tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.Tx5DhiB2Eq', fmt=raw size=1073741824 (finished: start all VMs, in 71.23 seconds) gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab: waiting for unit setup-gitlab.service gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for the VM to finish booting gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 8c17b31b-df1d-447a-b97b-864c71560f16 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] KASLR enabled gitlab # [ 0.000000] random: crng init done gitlab # [ 0.000000] Machine model: linux,dummy-virt gitlab # [ 0.000000] efi: UEFI not found. gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] gitlab # [ 0.000000] Zone ranges: gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] gitlab # [ 0.000000] DMA32 empty gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.000000] Device empty gitlab # [ 0.000000] Movable zone start for each node gitlab # [ 0.000000] Early memory node ranges gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 gitlab # [ 0.000000] psci: probing for conduit method from DT. gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs gitlab # [ 0.000000] psci: Trusted OS migration not required gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface gitlab # [ 0.000000] CPU features: detected: Spectre-v4 gitlab # [ 0.000000] CPU features: detected: Spectre-BHB gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 gitlab # [ 0.000000] alternatives: applying boot alternatives gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m0wkiq61cixn57g1dbzbfcnjcqz5077v-nixos-system-gitlab-test/init regInfo=/nix/store/0w02dgfdckidp2y6zc4wg1r7kj2a6d3c-closure-info/registration console=ttyAMA0,115200n8 console=tty0 gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0w02dgfdckidp2y6zc4wg1r7kj2a6d3c-closure-info/registration", will be passed to user space. gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.000000] software IO TLB: area num 4. gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) gitlab # [ 0.000000] Fallback order for Node 0: 0 gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 gitlab # [ 0.000000] Policy zone: Normal gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.000000] allocated 12582912 bytes of page_ext gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. gitlab # [ 0.000000] rcu: RCU event tracing is enabled. gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 gitlab # [ 0.000000] GICv3: 256 SPIs implemented gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f40000 gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f50000 gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns gitlab # [ 0.000029] arm-pv: using stolen time PV gitlab # [ 0.000369] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.000547] Console: colour dummy device 80x25 gitlab # [ 0.000555] printk: legacy console [tty0] enabled gitlab # [ 0.000739] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) gitlab # [ 0.000746] pid_max: default: 32768 minimum: 301 gitlab # [ 0.000833] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.000966] landlock: Up and running. gitlab # [ 0.000970] Yama: becoming mindful. gitlab # [ 0.001385] LSM support for eBPF active gitlab # [ 0.001581] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.001689] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.002849] cacheinfo: Unable to detect cache hierarchy for CPU 0 gitlab # [ 0.003555] rcu: Hierarchical SRCU implementation. gitlab # [ 0.003559] rcu: Max phase no-delay instances is 1000. gitlab # [ 0.003723] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.004799] fsl-mc MSI: its@8080000 domain created gitlab # [ 0.004894] EFI services will not be available. gitlab # [ 0.005113] smp: Bringing up secondary CPUs ... gitlab # [ 0.005649] Detected PIPT I-cache on CPU1 gitlab # [ 0.005717] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 gitlab # [ 0.005786] GICv3: CPU1: using allocated LPI pending table @0x0000000100f60000 gitlab # [ 0.005858] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] gitlab # [ 0.006882] Detected PIPT I-cache on CPU2 gitlab # [ 0.006991] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 gitlab # [ 0.007124] GICv3: CPU2: using allocated LPI pending table @0x0000000100f70000 gitlab # [ 0.007256] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] gitlab # [ 0.008308] Detected PIPT I-cache on CPU3 gitlab # [ 0.008424] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 gitlab # [ 0.008571] GICv3: CPU3: using allocated LPI pending table @0x0000000100f80000 gitlab # [ 0.008702] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] gitlab # [ 0.009196] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.009223] SMP: Total of 4 processors activated. gitlab # [ 0.009226] CPU: All CPU(s) started at EL1 gitlab # [ 0.009236] CPU features: detected: Branch Target Identification gitlab # [ 0.009240] CPU features: detected: ARMv8.4 Translation Table Level gitlab # [ 0.009243] CPU features: detected: Instruction cache invalidation not required for I/D coherence gitlab # [ 0.009246] CPU features: detected: Data cache clean to the PoU not required for I/D coherence gitlab # [ 0.009250] CPU features: detected: Common not Private translations gitlab # [ 0.009252] CPU features: detected: CRC32 instructions gitlab # [ 0.009255] CPU features: detected: Data cache clean to Point of Deep Persistence gitlab # [ 0.009258] CPU features: detected: Data cache clean to Point of Persistence gitlab # [ 0.009261] CPU features: detected: Data independent timing control (DIT) gitlab # [ 0.009264] CPU features: detected: E0PD gitlab # [ 0.009266] CPU features: detected: Enhanced Counter Virtualization gitlab # [ 0.009269] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) gitlab # [ 0.009272] CPU features: detected: Enhanced Virtualization Traps gitlab # [ 0.009274] CPU features: detected: Fine Grained Traps gitlab # [ 0.009278] CPU features: detected: Generic authentication (architected QARMA5 algorithm) gitlab # [ 0.009282] CPU features: detected: RCpc load-acquire (LDAPR) gitlab # [ 0.009284] CPU features: detected: LSE atomic instructions gitlab # [ 0.009287] CPU features: detected: Privileged Access Never gitlab # [ 0.009289] CPU features: detected: PMUv3 gitlab # [ 0.009292] CPU features: detected: RAS Extension Support gitlab # [ 0.009294] CPU features: detected: RASv1p1 Extension Support gitlab # [ 0.009297] CPU features: detected: Random Number Generator gitlab # [ 0.009299] CPU features: detected: Speculation barrier (SB) gitlab # [ 0.009302] CPU features: detected: Stage-2 Force Write-Back gitlab # [ 0.009304] CPU features: detected: TLB range maintenance instructions gitlab # [ 0.009308] CPU features: detected: Speculative Store Bypassing Safe (SSBS) gitlab # [ 0.009438] alternatives: applying system-wide alternatives gitlab # [ 0.012514] CPU features: detected: BBM Level 2 without TLB conflict abort gitlab # [ 0.012702] Memory: 5963764K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) gitlab # [ 0.013810] devtmpfs: initialized gitlab # [ 0.017370] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.017439] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.017615] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL gitlab # [ 0.017619] 0 pages in range for non-PLT usage gitlab # [ 0.017620] 508288 pages in range for PLT usage gitlab # [ 0.017756] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.018560] DMI not present or invalid. gitlab # [ 0.021834] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.024335] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.024643] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.026859] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.026882] audit: initializing netlink subsys (disabled) gitlab # [ 0.027181] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.027721] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.027724] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.027729] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.027731] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.027734] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.027814] cpuidle: using governor ladder gitlab # [ 0.027839] cpuidle: using governor menu gitlab # [ 0.028054] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. gitlab # [ 0.028262] ASID allocator initialised with 65536 entries gitlab # [ 0.029448] Serial: AMBA PL011 UART driver gitlab # [ 0.034496] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 gitlab # [ 0.034812] printk: console [ttyAMA0] enabled gitlab # [ 0.042312] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.042334] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.042344] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages gitlab # [ 0.042352] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page gitlab # [ 0.042361] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.042369] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.042377] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages gitlab # [ 0.042385] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page gitlab # [ 0.050963] fbcon: Taking over console gitlab # [ 0.050994] ACPI: Interpreter disabled. gitlab # [ 0.053917] iommu: Default domain type: Translated gitlab # [ 0.053935] iommu: DMA domain TLB invalidation policy: strict mode gitlab # [ 0.062131] SCSI subsystem initialized gitlab # [ 0.063127] usbcore: registered new interface driver usbfs gitlab # [ 0.063195] usbcore: registered new interface driver hub gitlab # [ 0.063239] usbcore: registered new device driver usb gitlab # [ 0.064130] pps_core: LinuxPPS API ver. 1 registered gitlab # [ 0.064141] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti gitlab # [ 0.064194] PTP clock support registered gitlab # [ 0.064301] EDAC MC: Ver: 3.0.0 gitlab # [ 0.064949] scmi_core: SCMI protocol bus registered gitlab # [ 0.066075] FPGA manager framework gitlab # [ 0.068447] vgaarb: loaded gitlab # [ 0.073899] clocksource: Switched to clocksource arch_sys_counter gitlab # [ 0.077000] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.077065] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.078024] netfs: FS-Cache loaded gitlab # [ 0.078528] pnp: PnP ACPI: disabled gitlab # [ 0.087613] NET: Registered PF_INET protocol family gitlab # [ 0.090390] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.149306] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.149373] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.149431] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.149815] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.150697] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.150846] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.150930] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.151097] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.151345] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.151384] NET: Registered PF_XDP protocol family gitlab # [ 0.151404] PCI: CLS 0 bytes, default 64 gitlab # [ 0.151682] Trying to unpack rootfs image as initramfs... gitlab # [ 0.162729] kvm [1]: HYP mode not available gitlab # [ 0.230664] Initialise system trusted keyrings gitlab # [ 0.231071] workingset: timestamp_bits=42 max_order=21 bucket_order=0 gitlab # [ 0.232334] squashfs: version 4.0 (2009/01/31) Phillip Lougher gitlab # [ 0.232552] 9p: Installing v9fs 9p2000 file system support gitlab # [ 0.270915] Key type asymmetric registered gitlab # [ 0.270949] Asymmetric key parser 'x509' registered gitlab # [ 0.271119] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) gitlab # [ 0.271549] io scheduler mq-deadline registered gitlab # [ 0.271568] io scheduler kyber registered gitlab # [ 0.286733] pl061_gpio 9030000.pl061: PL061 GPIO chip registered gitlab # [ 0.290124] ledtrig-cpu: registered to indicate activity on CPUs gitlab # [ 0.291202] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: gitlab # [ 0.291239] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 gitlab # [ 0.291267] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 gitlab # [ 0.291286] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 gitlab # [ 0.291333] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits gitlab # [ 0.291386] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] gitlab # [ 0.291589] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 gitlab # [ 0.291608] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.291620] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] gitlab # [ 0.291629] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] gitlab # [ 0.291638] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] gitlab # [ 0.291804] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.293241] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.293850] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.293950] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.294035] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.294072] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.295821] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.296554] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.296591] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.296672] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.298386] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.299080] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.299117] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.299227] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.300933] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.301621] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.301657] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.301736] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.303383] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.304030] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.304066] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.304189] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.305946] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.306675] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.306711] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.306827] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.308436] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.309036] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.309073] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.309167] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.309204] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.311011] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint gitlab # [ 0.311707] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.311789] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.313516] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.314345] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.314428] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.315798] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint gitlab # [ 0.316516] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] gitlab # [ 0.317440] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.318253] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] gitlab # [ 0.318290] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.318371] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.320174] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.320781] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.320819] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.320919] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.323052] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned gitlab # [ 0.323069] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned gitlab # [ 0.323080] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned gitlab # [ 0.323216] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned gitlab # [ 0.323353] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned gitlab # [ 0.323486] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned gitlab # [ 0.323629] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned gitlab # [ 0.323773] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned gitlab # [ 0.323916] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned gitlab # [ 0.324084] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned gitlab # [ 0.324234] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned gitlab # [ 0.324381] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned gitlab # [ 0.324573] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned gitlab # [ 0.324705] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned gitlab # [ 0.324773] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned gitlab # [ 0.324828] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned gitlab # [ 0.324880] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned gitlab # [ 0.324947] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned gitlab # [ 0.325001] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned gitlab # [ 0.325067] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned gitlab # [ 0.325122] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned gitlab # [ 0.325178] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned gitlab # [ 0.325248] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned gitlab # [ 0.325306] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned gitlab # [ 0.325361] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned gitlab # [ 0.325415] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned gitlab # [ 0.325479] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned gitlab # [ 0.325535] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned gitlab # [ 0.325589] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned gitlab # [ 0.325643] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned gitlab # [ 0.325696] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned gitlab # [ 0.325749] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned gitlab # [ 0.325806] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned gitlab # [ 0.325861] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned gitlab # [ 0.325970] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] gitlab # [ 0.325986] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] gitlab # [ 0.325995] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] gitlab # [ 0.330221] pci 0000:00:0a.0: enabling device (0000 -> 0002) gitlab # [ 0.366411] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) gitlab # [ 0.372342] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) gitlab # [ 0.378469] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) gitlab # [ 0.385271] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) gitlab # [ 0.391386] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) gitlab # [ 0.398338] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) gitlab # [ 0.404699] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) gitlab # [ 0.411820] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) gitlab # [ 0.417734] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) gitlab # [ 0.424209] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) gitlab # [ 0.430996] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) gitlab # [ 0.447631] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.452736] msm_serial: driver initialized gitlab # [ 0.453152] SuperH (H)SCI(F) driver initialized gitlab # [ 0.453327] STM32 USART driver initialized gitlab # [ 0.514055] loop: module loaded gitlab # [ 0.514600] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 0.519820] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 0.524580] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 0.529749] virtio_blk virtio5: [vdb] 12286184 512-byte logical blocks (6.29 GB/5.86 GiB) gitlab # [ 0.535293] megasas: 07.734.00.00-rc1 gitlab # [ 0.537346] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] gitlab # [ 0.542732] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.542769] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.547841] Using buffer write method gitlab # [ 0.547912] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] gitlab # [ 0.554518] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.554552] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.558461] Using buffer write method gitlab # [ 0.558495] Concatenating MTD devices: gitlab # [ 0.558502] (0): "0.flash" gitlab # [ 0.558508] (1): "0.flash" gitlab # [ 0.558513] into device "0.flash" gitlab # [ 0.662875] Freeing initrd memory: 27184K gitlab # [ 0.682891] tun: Universal TUN/TAP device driver, 1.6 gitlab # [ 0.691834] thunder_xcv, ver 1.0 gitlab # [ 0.691945] thunder_bgx, ver 1.0 gitlab # [ 0.692009] nicpf, ver 1.0 gitlab # [ 0.693800] e1000: Intel(R) PRO/1000 Network Driver gitlab # [ 0.693812] e1000: Copyright (c) 1999-2006 Intel Corporation. gitlab # [ 0.693979] e1000e: Intel(R) PRO/1000 Network Driver gitlab # [ 0.693994] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. gitlab # [ 0.694067] igb: Intel(R) Gigabit Ethernet Network Driver gitlab # [ 0.694077] igb: Copyright (c) 2007-2014 Intel Corporation. gitlab # [ 0.694138] igbvf: Intel(R) Gigabit Virtual Function Network Driver gitlab # [ 0.694149] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. gitlab # [ 0.694623] sky2: driver version 1.30 gitlab # [ 0.699785] usbcore: registered new interface driver usb-storage gitlab # [ 0.700098] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.700139] usbserial: USB Serial support registered for generic gitlab # [ 0.700613] ehci-pci 0000:00:0a.0: EHCI Host Controller gitlab # [ 0.700668] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 gitlab # [ 0.701385] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 gitlab # [ 0.702043] hv_vmbus: registering driver hyperv_keyboard gitlab # [ 0.704793] rtc-pl031 9010000.pl031: registered as rtc0 gitlab # [ 0.704855] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T19:17:31 UTC (1790191051) gitlab # [ 0.705781] i2c_dev: i2c /dev entries driver gitlab # [ 0.709997] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 gitlab # [ 0.711342] hub 1-0:1.0: USB hub found gitlab # [ 0.711570] hub 1-0:1.0: 6 ports detected gitlab # [ 0.714474] sdhci: Secure Digital Host Controller Interface driver gitlab # [ 0.714489] sdhci: Copyright(c) Pierre Ossman gitlab # [ 0.715340] Synopsys Designware Multimedia Card Interface Driver gitlab # [ 0.716513] sdhci-pltfm: SDHCI platform and OF driver helper gitlab # [ 0.721377] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 0.722272] usbcore: registered new interface driver usbhid gitlab # [ 0.722284] usbhid: USB HID core driver gitlab # [ 0.759685] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available gitlab # [ 0.764530] drop_monitor: Initializing network drop monitor service gitlab # [ 0.764959] NET: Registered PF_INET6 protocol family gitlab # [ 0.766946] Segment Routing with IPv6 gitlab # [ 0.766981] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.767046] NET: Registered PF_PACKET protocol family gitlab # [ 0.767301] 9pnet: Installing 9P2000 support gitlab # [ 0.772043] Key type dns_resolver registered gitlab # [ 0.792039] registered taskstats version 1 gitlab # [ 0.792777] Loading compiled-in X.509 certificates gitlab # [ 0.815583] Demotion targets for Node 0: null gitlab # [ 0.815820] Key type .fscrypt registered gitlab # [ 0.815828] Key type fscrypt-provisioning registered gitlab # [ 0.816020] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.816043] ima: Allocated hash algorithm: sha1 gitlab # [ 0.816092] ima: No architecture policies found gitlab # [ 0.817514] input: gpio-keys as /devices/platform/gpio-keys/input/input0 gitlab # [ 0.844303] clk: Disabling unused clocks gitlab # [ 0.844340] PM: genpd: Disabling unused power domains gitlab # [ 0.849840] Freeing unused kernel memory: 4736K gitlab # [ 0.850099] Run /init as init process gitlab # [ 0.888573] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 0.982040] usb 1-1: new high-speed USB device number 2 using ehci-pci gitlab # [ 1.138698] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 gitlab # [ 1.223110] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.223180] systemd[1]: Detected virtualization qemu. gitlab # [ 1.223256] systemd[1]: Detected architecture arm64. gitlab # [ 1.223271] systemd[1]: Running in initrd. gitlab # [ 1.224139] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.224397] systemd[1]: Hostname set to . gitlab # [ 1.246713] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 gitlab # [ 1.365946] usb 1-2: new high-speed USB device number 3 using ehci-pci gitlab # [ 1.437267] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.504038] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.520089] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 gitlab # [ 1.520660] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 gitlab # [ 1.521083] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.521322] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.521355] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.521369] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.521392] systemd[1]: Reached target Path Units. gitlab # [ 1.521408] systemd[1]: Reached target Slice Units. gitlab # [ 1.521427] systemd[1]: Reached target Swaps. gitlab # [ 1.521445] systemd[1]: Reached target Timer Units. gitlab # [ 1.521615] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.521816] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.522124] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.522277] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.522369] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.522390] systemd[1]: Reached target Socket Units. gitlab # [ 1.524235] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.526132] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.526204] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 1.527932] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 1.557652] systemd[1]: Starting Journal Service... gitlab # [ 1.559731] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.559841] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.561337] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.567248] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.568421] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.568806] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.569090] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 1.574402] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.605699] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 gitlab # [ 1.607687] [drm] features: -virgl +edid -resource_blob -host_visible gitlab # [ 1.607716] [drm] features: -context_init gitlab # [ 1.609238] [drm] number of scanouts: 1 gitlab # [ 1.609273] [drm] number of cap sets: 0 gitlab # [ 1.609549] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.609968] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.618506] systemd-journald[96]: Collecting audit messages is disabled. gitlab # [ 1.622476] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic gitlab # [ 1.622518] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 gitlab # [ 1.634306] Console: switching to colour frame buffer device 160x50 gitlab # [ 1.641048] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device gitlab # [ 1.647812] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 1.651225] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 1.670467] systemd[1]: Finished Load Kernel Modules. gitlab # [ 1.678199] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.689364] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.689779] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.689827] systemd[1]: Reached target Local File Systems. gitlab # [ 1.692172] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.703485] systemd[1]: Started Journal Service. gitlab # [ 1.693985] systemd-modules-load[97]: Using 4 probe threads gitlab # [ 1.705553] systemd-modules-load[97]: Module 'virtio_console' is built in gitlab # [ 1.706752] systemd-modules-load[97]: Module 'virtio_rng' is built in gitlab # [ 1.707853] systemd-modules-load[97]: Module 'virtio_balloon' is built in gitlab # [ 1.709059] systemd-modules-load[97]: Inserted module 'dm_mod' gitlab # [ 1.710013] systemd-modules-load[97]: Inserted module 'virtio_gpu' gitlab # [ 1.711019] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.716252] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.729198] systemd-udevd[109]: Using default interface naming scheme 'v261'. gitlab # [ 1.738203] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.745608] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.761972] systemd[1]: Starting Virtual Console Setup... gitlab # [ 1.827307] systemd-vconsole-setup[131]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 1.831986] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.214638] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.217482] systemd[1]: Reached target System Initialization. gitlab # [ 2.219948] systemd[1]: Reached target Basic System. gitlab # [ 2.433917] (udev-worker)[127]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.440102] (udev-worker)[127]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.452076] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.461177] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.464764] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.468638] (udev-worker)[124]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.500614] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.521997] systemd-fsck[153]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.529423] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.562410] systemd[1]: Mounting /sysroot... gitlab # [ 2.601285] EXT4-fs (vda): mounted filesystem 8c17b31b-df1d-447a-b97b-864c71560f16 r/w with ordered data mode. Quota mode: none. gitlab # [ 2.602743] systemd[1]: Mounted /sysroot. gitlab # [ 2.605618] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.610533] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.616382] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.632592] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.633759] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 2.640355] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 2.645055] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 2.658391] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 2.662176] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 2.664598] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 2.668054] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 2.670816] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 2.672289] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 2.677419] systemd[1]: Mounted /sysroot/run. gitlab # [ 2.680208] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 2.682398] systemd[1]: Reloading... gitlab # [ 2.790013] systemd[1]: Reloading finished in 108 ms. gitlab # [ 2.844281] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace... gitlab # [ 2.847172] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace... gitlab # [ 2.849376] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.852936] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 2.854183] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 2.855813] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 2.880758] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 2.881948] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.013286] (udev-worker)[122]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory gitlab # [ 3.015936] (udev-worker)[122]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory gitlab # [ 3.024658] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.025754] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.026500] systemd[1]: Stopping Virtual Console Setup... gitlab # [ 3.027287] systemd[1]: Starting Virtual Console Setup... gitlab # [ 3.044833] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.045938] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.046688] systemd[1]: Starting Virtual Console Setup... gitlab # [ 3.067796] systemd-vconsole-setup[195]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 3.069799] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.568422] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.639004] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.641378] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.643813] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.646747] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.687910] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.691488] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.711549] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.713983] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.716695] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.770318] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.773040] systemd[1]: Stopped target Basic System. gitlab # [ 3.775209] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.777699] systemd[1]: Stopped target Path Units. gitlab # [ 3.779814] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.783160] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.786510] systemd[1]: Stopped target Slice Units. gitlab # [ 3.789454] systemd[1]: Stopped target Socket Units. gitlab # [ 3.794577] systemd[1]: Stopped target System Initialization. gitlab # [ 3.802667] systemd[1]: Stopped target Swaps. gitlab # [ 3.806329] systemd[1]: Stopped target Timer Units. gitlab # [ 3.808837] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.811678] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.814382] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.817992] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.819786] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 3.822301] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.824225] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.826401] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.828157] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.830257] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.831719] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.834071] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.836351] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.838397] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.840389] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.842047] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.844067] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.846097] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.847693] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.849922] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.851991] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.853512] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.855427] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.857422] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.859400] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.861023] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 3.862949] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 3.864507] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.866151] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.867885] systemd[1]: systemd-udevd.service: Consumed 2.249s CPU time over 2.152s wall clock time, 31.2M memory peak. gitlab # [ 3.870433] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.872089] systemd[1]: Closed udev Control Socket. gitlab # [ 3.873224] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.874458] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.876130] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.877481] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.879188] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.880688] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.882110] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 3.968904] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 3.972220] systemd[1]: Finished Cleanup udev Database. gitlab # [ 3.974524] systemd[1]: Reached target Switch Root. gitlab # [ 3.976736] systemd[1]: Starting NixOS Activation... gitlab # [ 4.149879] initrd-nixos-activation-start[221]: booting system configuration /nix/store/m0wkiq61cixn57g1dbzbfcnjcqz5077v-nixos-system-gitlab-test gitlab # [ 4.216875] initrd-nixos-activation-start[221]: running activation script... gitlab # [ 4.550944] initrd-nixos-activation-start[244]: setting up /etc... gitlab # [ 4.707027] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.711567] systemd[1]: Finished NixOS Activation. gitlab # [ 4.713816] systemd[1]: Starting Switch Root... gitlab # [ 4.753231] systemd[1]: Switching root. gitlab # [ 4.854429] systemd-journald[96]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.454941] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.455092] systemd[1]: Detected virtualization qemu. gitlab # [ 5.455175] systemd[1]: Detected architecture arm64. gitlab # [ 5.455378] systemd[1]: Detected first boot. gitlab # [ 5.456857] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.713875] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.828271] systemd[1]: Applying preset policy. gitlab # [ 6.010888] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.275403] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.275949] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.278944] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.283556] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.289824] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.293534] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.294579] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.295861] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.296750] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.297468] systemd[1]: Expecting device /dev/ttyAMA0... gitlab # [ 6.298483] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.299590] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.300681] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.301737] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.302914] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.303896] systemd[1]: Reached target Path Units. gitlab # [ 6.304699] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.305399] systemd[1]: Reached target Slice Units. gitlab # [ 6.306721] systemd[1]: Reached target Swaps. gitlab # [ 6.311575] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 6.316460] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.319988] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.324216] systemd[1]: Listening on Factory Reset Management. gitlab # [ 6.325767] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 6.336561] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.339054] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.343701] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.345485] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.346706] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.347748] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.355149] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.356595] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.358002] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 6.367052] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.377581] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.402897] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.409129] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.418926] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.434235] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 6.435267] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 6.445771] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.446721] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.447508] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.463246] systemd[1]: Starting Load Kernel Module fuse... gitlab # [ 6.464110] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.475514] systemd[1]: Starting Journal Service... gitlab # [ 6.482593] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.494259] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.499414] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.500927] fuse: init (API version 7.45) gitlab # [ 6.502594] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.510104] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.517333] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.518115] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.518982] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.519838] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.520795] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.522157] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.523347] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 6.524381] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 6.525360] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.526410] systemd[1]: modprobe@fuse.service: Deactivated successfully. gitlab # [ 6.527438] systemd[1]: Finished Load Kernel Module fuse. gitlab # [ 6.528893] systemd[1]: Finished Load Kernel Modules. gitlab # [ 6.538627] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.550095] systemd[1]: Starting Firewall... gitlab # [ 6.558106] EXT4-fs (vda): re-mounted 8c17b31b-df1d-447a-b97b-864c71560f16. gitlab # [ 6.564802] systemd-journald[315]: Collecting audit messages is enabled. gitlab # [ 6.579303] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.579720] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 6.584424] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.585553] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.586809] systemd-modules-load[316]: Using 3 probe threads gitlab # [ 6.590074] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.587972] systemd-modules-load[316]: Module 'atkbd' is built in gitlab # [ 6.593449] systemd-modules-load[316]: Module 'loop' is built in gitlab # [ 6.592039] systemd[1]: Started Journal Service. gitlab # [ 6.598463] systemd-modules-load[316]: Inserted module 'tls' gitlab # [ 6.608099] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.611857] systemd-oomd[317]: No swap; memory pressure usage will be degraded gitlab # [ 6.613926] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.614907] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.621740] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.633677] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.635675] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.638672] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.641081] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.643060] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.664653] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.666100] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.677509] systemd-journald[315]: Received client request to flush runtime journal. gitlab # [ 6.699266] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.700827] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.701726] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.702719] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.703789] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.704901] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.751939] systemd-udevd[373]: Using default interface naming scheme 'v261'. gitlab # [ 6.796918] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.987499] systemd[1]: Finished Firewall. gitlab # [ 7.201652] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 7.287798] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.346308] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 7.371560] systemd[1]: Mounted /run/wrappers. gitlab # [ 7.375753] systemd[1]: Reached target Local File Systems. gitlab # [ 7.382125] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 7.396201] systemd[1]: Starting register-nix-paths.service... gitlab # [ 7.413844] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 7.416554] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.428241] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 7.436611] systemd[1]: Starting Create System Files and Directories... gitlab # [ 7.483308] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 7.488184] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.504671] systemd[1]: Finished Create System Files and Directories. gitlab # [ 7.510328] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 7.518991] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 7.546260] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 7.572557] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.592659] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.596441] systemd[1]: Starting Update is Completed... gitlab # [ 7.601162] (udev-worker)[473]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.608968] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. gitlab # [ 7.622408] (udev-worker)[432]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 7.624851] (udev-worker)[432]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.634157] systemd[1]: Finished Update is Completed. gitlab # [ 7.643228] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 7.645049] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.647281] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.649578] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.653469] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.655974] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.657349] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.698191] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.699418] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.700795] systemd[1]: suid-sgid-wrappers.service: Consumed 181ms CPU time over 284ms wall clock time, 6.3M memory peak, 1.9M read from disk. gitlab # [ 7.718739] mousedev: PS/2 mouse device common for all mice gitlab # [ 7.923132] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.924402] systemd[1]: Reached target System Initialization. gitlab # [ 7.925530] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.926916] systemd[1]: Started logrotate.timer. gitlab # [ 7.927663] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.929651] systemd[1]: Reached target Timer Units. gitlab # [ 7.930707] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.931920] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.941342] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.943149] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.945659] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.946924] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.947939] systemd[1]: Reached target Socket Units. gitlab # [ 7.982489] dbus-broker-launch[561]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.983903] dbus-broker-launch[561]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.985440] dbus-broker-launch[561]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.987976] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 8.003354] dbus-broker-launch[561]: Ready gitlab # [ 8.004240] systemd[1]: Reached target Basic System. gitlab # [ 8.006391] systemd[1]: Started backdoor.service. gitlab # [ 8.010771] systemd[1]: Starting gitlab-config.service... gitlab # [ 8.026418] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 8.043290] systemd[1]: Starting Logrotate configuration check... gitlab # [ 8.044714] logrotate[566]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 8.047823] logrotate[566]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf gitlab # [ 8.050020] logrotate[566]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 8.051228] logrotate[566]: Reading state from file: /var/lib/logrotate.status gitlab # [ 8.052505] logrotate[566]: state file /var/lib/logrotate.status does not exist gitlab # [ 8.053515] logrotate[566]: Allocating hash table for state file, size 64 entries gitlab # [ 8.054461] logrotate[566]: Handling 4 logs gitlab # connecting to host... gitlab # [ 8.055090] logrotate[566]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 8.057509] logrotate[566]: considering log /var/log/btmp gitlab # [ 8.058303] logrotate[566]: Creating new state gitlab # [ 8.059057] logrotate[566]: Now: 2026-09-23 19:17 gitlab # [ 8.059802] logrotate[566]: Last rotated at 2026-09-23 19:00 gitlab # [ 8.063363] logrotate[566]: log does not need rotating (log has already been rotated) gitlab # [ 8.064567] logrotate[566]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 8.067082] logrotate[566]: considering log /var/log/wtmp gitlab # [ 8.068106] logrotate[566]: Creating new state gitlab # [ 8.068847] logrotate[566]: Now: 2026-09-23 19:17 gitlab # [ 8.069534] logrotate[566]: Last rotated at 2026-09-23 19:00 gitlab # [ 8.070296] logrotate[566]: log does not need rotating (log has already been rotated) gitlab # [ 8.071330] logrotate[566]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 8.074274] logrotate[566]: switching euid from 0 to 165 and egid from 0 to 165 (pid 566) gitlab # [ 8.078450] logrotate[566]: considering log /var/gitlab/state/log/*.log gitlab # [ 8.079470] logrotate[566]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 8.083679] logrotate[566]: Creating new state gitlab # [ 8.084410] logrotate[566]: switching euid from 165 to 0 and egid from 165 to 0 (pid 566) gitlab: connected to guest root shell gitlab # [ 8.085523] logrotate[566]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab: (connecting took 8.54 seconds) gitlab # [ 8.087090] logrotate[566]: switching euid from 0 to 60 and egid from 0 to 60 (pid 566) gitlab: (finished: waiting for the VM to finish booting, in 8.54 seconds) gitlab # [ 8.088352] logrotate[566]: considering log /var/log/nginx/*.log gitlab # [ 8.089103] logrotate[566]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 8.089992] logrotate[566]: Creating new state gitlab # [ 8.090548] logrotate[566]: not running postrotate script, since no logs were rotated gitlab # [ 8.091819] logrotate[566]: switching euid from 60 to 0 and egid from 60 to 0 (pid 566) gitlab # [ 8.093019] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.093920] systemd[1]: Starting Post-Boot Actions... gitlab # [ 8.094731] systemd[1]: Started Reset console on configuration changes. gitlab # [ 8.096062] nsncd[570]: Sep 23 19:17:38.885 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.098611] systemd[1]: Starting resolvconf update... gitlab # [ 8.099471] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.100462] systemd[1]: Finished Logrotate configuration check. gitlab # [ 8.101898] systemd[1]: Finished Post-Boot Actions. gitlab # [ 8.105592] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 8.109577] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.110561] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.114027] systemd[1]: Starting User Login Management... gitlab # [ 8.177182] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 8.178847] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 8.179779] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 8.184098] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 8.184948] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 8.187054] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 8.190370] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 8.191306] systemd[1]: nscd.service: Consumed 23ms CPU time over 143ms wall clock time, 7.5M memory peak, 5.8M read from disk. gitlab # [ 8.203263] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.247667] systemd[1]: Finished resolvconf update. gitlab # [ 8.250848] systemd[1]: Reached target Preparation for Network. gitlab # [ 8.265372] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 gitlab # [ 8.263580] systemd-logind[604]: New seat seat0. gitlab # [ 8.339939] systemd[1]: Starting DHCP Client... gitlab # [ 8.347482] nsncd[661]: Sep 23 19:17:39.060 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.407087] systemd[1]: Starting Address configuration of eth1... gitlab # [ 8.411141] systemd[1]: Starting Extra networking commands.... gitlab # [ 8.412608] network-addresses-eth1-start[682]: adding address 192.168.1.1/24... done gitlab # [ 8.413724] network-addresses-eth1-start[682]: adding address 2001:db8:1::1/64... done gitlab # [ 8.414832] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.415862] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.426933] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.428032] systemd[1]: Finished Address configuration of eth1. gitlab # [ 8.428947] systemd-logind[604]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) gitlab # [ 8.431028] systemd-logind[604]: Watching system buttons on /dev/input/event0 (gpio-keys) gitlab # [ 8.432282] systemd[1]: Started User Login Management. gitlab # [ 8.433056] systemd[1]: Starting linger-users.service... gitlab # [ 8.448486] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 8.449628] systemd[1]: Finished linger-users.service. gitlab # [ 8.471813] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.502603] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 8.503671] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 8.504787] dhcpcd[702]: dhcpcd-10.3.2 starting gitlab # [ 8.507138] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.508960] dhcpcd[742]: dev: loaded udev gitlab # [ 8.524870] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 8.525262] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.531625] systemd-logind[604]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 8.539618] systemd[1]: Finished Extra networking commands.. gitlab # [ 8.542161] systemd[1]: Reached target Network. gitlab # [ 8.543491] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.548128] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 8.551424] systemd[1]: Starting Nginx Web Server... gitlab # [ 8.563852] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 8.562808] systemd[1]: Starting PostgreSQL Server... gitlab # [ 8.576486] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 8.576950] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 8.578568] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 8.577601] systemd[1]: Starting Redis Server - redis-gitlab...[ 8.578872] cfg80211: failed to load regulatory.db gitlab # gitlab # [ 8.582227] systemd-vconsole-setup[743]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 8.585174] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.586371] systemd[1]: Finished Virtual Console Setup. gitlab # [ 8.613132] systemd[1]: Finished Permit User Sessions. gitlab # [ 8.619407] systemd[1]: Started Getty on tty1. gitlab # [ 8.620309] systemd[1]: Reached target Login Prompts.[ 8.624575] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # gitlab # [ 8.624847] dhcpcd[742]: eth0: waiting for carrier gitlab # [ 8.625715] dhcpcd[742]: libudev: received NULL device gitlab # [ 8.626427] dhcpcd[742]: libudev: received NULL device gitlab # [ 8.627281] dhcpcd[742]: eth0: carrier acquired gitlab # [ 8.630553] dhcpcd[742]: DUID 00:01:00:01:32:46:e2:53:52:54:00:12:34:56 gitlab # [ 8.631563] dhcpcd[742]: eth0: IAID 00:12:34:56 gitlab # [ 8.633076] dhcpcd[742]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 8.717134] nginx-pre-start[792]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok gitlab # [ 8.718910] nginx-pre-start[792]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful gitlab # [ 8.724962] systemd[1]: Started Nginx Web Server. gitlab # [ 8.753405] postgresql-pre-start[796]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.754938] postgresql-pre-start[796]: This user must also own the server process. gitlab # [ 8.756957] redis-server[793]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 8.759074] redis-server[793]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 8.760219] postgresql-pre-start[796]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.763122] postgresql-pre-start[796]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.764723] postgresql-pre-start[796]: The default text search configuration will be set to "english". gitlab # [ 8.766432] postgresql-pre-start[796]: Data page checksums are enabled. gitlab # [ 8.767703] postgresql-pre-start[796]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 8.769735] postgresql-pre-start[796]: creating subdirectories ... ok gitlab # [ 8.770885] redis-server[793]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=793, just started gitlab # [ 8.772663] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 8.773642] postgresql-pre-start[796]: selecting dynamic shared memory implementation ... posix gitlab # [ 8.775127] redis-server[793]: Configuration loaded gitlab # [ 8.776142] redis[793]: monotonic clock: ARM CNTVCT @ 1000 ticks/us gitlab # [ 8.777227] redis[793]: Running mode=standalone, port=0. gitlab # [ 8.778084] redis[793]: Server initialized gitlab # [ 8.778851] redis[793]: Ready to accept connections unix gitlab # [ 8.779693] redis[793]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 8.800649] postgresql-pre-start[796]: selecting default "max_connections" ... 100 gitlab # [ 8.849915] postgresql-pre-start[796]: selecting default "shared_buffers" ... 128MB gitlab # [ 9.153339] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 9.394772] gitlab-workhorse[819]: time="2026-09-23T19:17:40Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 9.396692] gitlab-workhorse[819]: time="2026-09-23T19:17:40Z" level=info msg="Using redis/go-redis" gitlab # [ 9.398086] gitlab-workhorse[819]: time="2026-09-23T19:17:40Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 9.407573] postgresql-pre-start[796]: selecting default time zone ... UTC gitlab # [ 9.409464] postgresql-pre-start[796]: creating configuration files ... ok gitlab # [ 9.499200] dhcpcd[742]: eth0: soliciting a DHCP lease gitlab # [ 9.513623] dhcpcd[742]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.532430] dhcpcd[742]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.638952] postgresql-pre-start[796]: running bootstrap script ... ok gitlab # [ 10.238961] dhcpcd[742]: eth0: soliciting an IPv6 router gitlab # [ 10.244465] dhcpcd[742]: eth0: Router Advertisement from fe80::2 gitlab # [ 10.247156] dhcpcd[742]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 10.252450] dhcpcd[742]: eth0: adding route to fec0::/64 gitlab # [ 10.254832] dhcpcd[742]: eth0: adding default route via fe80::2 gitlab # [ 10.358232] postgresql-pre-start[796]: performing post-bootstrap initialization ... ok gitlab # [ 10.643677] postgresql-pre-start[796]: syncing data to disk ... ok gitlab # [ 10.646636] postgresql-pre-start[796]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 10.650789] postgresql-pre-start[796]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 10.657205] postgresql-pre-start[796]: Success. You can now start the database server using: gitlab # [ 10.660706] postgresql-pre-start[796]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 10.863832] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 10.869954] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 10.873367] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 10.878383] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 10.884271] postgres[868]: [868] LOG: database system was shut down at 2026-09-23 19:17:41 GMT gitlab # [ 10.891527] postgres[854]: [854] LOG: database system is ready to accept connections gitlab # [ 10.896475] systemd[1]: Started PostgreSQL Server. gitlab # [ 10.902973] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.928428] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[861]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 10.932568] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[861]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 11.020583] systemd[1]: Finished gitlab-config.service. gitlab # [ 11.027172] systemd[1]: Started gitaly.service. gitlab # [ 11.232410] postgresql-setup-start[890]: CREATE DATABASE gitlab # [ 11.290430] postgresql-setup-start[904]: CREATE ROLE gitlab # [ 11.313570] postgresql-setup-start[906]: ALTER DATABASE gitlab # [ 11.371996] postgresql-setup-start[915]: CREATE ROLE gitlab # [ 11.376399] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 11.381305] systemd[1]: Reached target PostgreSQL. gitlab # [ 11.388165] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 11.390143] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 11.580510] gitlab-postgresql-start[930]: CREATE DATABASE gitlab # [ 11.610111] gitaly[892]: time="2026-09-23T19:17:42.404Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=892 gitlab # [ 11.628689] gitaly[892]: time="2026-09-23T19:17:42.404Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=892 gitlab # [ 11.636544] gitaly[892]: time="2026-09-23T19:17:42.404Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=892 gitlab # [ 11.645851] gitaly[892]: time="2026-09-23T19:17:42.405Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=892 gitlab # [ 11.678795] gitaly[892]: time="2026-09-23T19:17:42.405Z" level=info msg="Starting Gitaly" pid=892 version= gitlab # [ 11.682367] gitlab-postgresql-start[936]: CREATE EXTENSION gitlab # [ 11.800279] gitlab-postgresql-start[939]: CREATE EXTENSION gitlab # [ 11.802208] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 11.853119] gitaly[892]: time="2026-09-23T19:17:42.649Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=244 pid=892 gitlab # [ 11.855049] gitaly[892]: time="2026-09-23T19:17:42.652Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=892 gitlab # [ 11.856623] gitaly[892]: time="2026-09-23T19:17:42.653Z" level=info msg="finished initializing command factory" duration_ms=0 pid=892 gitlab # [ 11.858127] gitaly[892]: time="2026-09-23T19:17:42.653Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-892/git-exec-112072439.d/git pid=892 gitlab # [ 11.865210] gitaly[892]: time="2026-09-23T19:17:42.662Z" level=info msg="finished detecting git version" duration_ms=8 pid=892 gitlab # [ 11.866833] gitaly[892]: time="2026-09-23T19:17:42.662Z" level=info msg="using Git version" pid=892 version=2.53.g gitlab # [ 11.868261] gitaly[892]: time="2026-09-23T19:17:42.662Z" level=info msg="clearing disk cache object folder" pid=892 storage=default gitlab # [ 11.869750] gitaly[892]: time="2026-09-23T19:17:42.662Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1327600921 pid=892 storage=default gitlab # [ 11.872258] gitaly[892]: time="2026-09-23T19:17:42.662Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1327600921 pid=892 storage=default gitlab # [ 11.874874] gitaly[892]: time="2026-09-23T19:17:42.662Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=892 gitlab # [ 11.877396] gitaly[892]: time="2026-09-23T19:17:42.662Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=892 gitlab # [ 11.880200] gitaly[892]: time="2026-09-23T19:17:42.663Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1327600921 clear_duration_ms=0 pid=892 storage=default gitlab # [ 11.888001] gitaly[892]: time="2026-09-23T19:17:42.684Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=892 gitlab # [ 11.890440] gitaly[892]: time="2026-09-23T19:17:42.685Z" level=info msg="finished tempdir cleaner walk" pid=892 storage=default time_ms=0 gitlab # [ 11.892127] gitaly[892]: time="2026-09-23T19:17:42.685Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=892 schema=unix gitlab # [ 11.894028] gitaly[892]: time="2026-09-23T19:17:42.685Z" level=info msg="listening at address" address=/run/gitaly/gitaly-892/sock.d/intern pid=892 schema=unix gitlab # [ 11.895829] gitaly[892]: time="2026-09-23T19:17:42.684Z" level=info msg="counting repositories" pid=892 gitlab # [ 11.897121] gitaly[892]: time="2026-09-23T19:17:42.687Z" level=info msg="starting to count repositories" pid=892 storage_path=/var/gitlab/state/repositories gitlab # [ 11.898887] gitaly[892]: time="2026-09-23T19:17:42.687Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=892 storage_path=/var/gitlab/state/repositories gitlab # [ 11.900998] gitaly[892]: time="2026-09-23T19:17:42.687Z" level=info msg="completed counting all repositories" counting_duration=0 pid=892 gitlab # [ 11.902641] gitaly[892]: time="2026-09-23T19:17:42.688Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=892 gitlab # [ 11.904230] gitaly[892]: time="2026-09-23T19:17:42.688Z" level=info msg="Started Gitaly" duration_ms=283 pid=892 gitlab # [ 11.905539] gitaly[892]: time="2026-09-23T19:17:42.688Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=892 gitlab # [ 11.907055] gitaly[892]: time="2026-09-23T19:17:42.688Z" level=info msg="maintenance: daily scheduled" pid=892 scheduled="2026-09-24 12:00:00 +0000 UTC" gitlab # [ 12.639397] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[925]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.647635] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[925]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.675052] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[925]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 14.007444] dhcpcd[742]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.007751] dhcpcd[742]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.007965] dhcpcd[742]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.197569] systemd[1]: Started DHCP Client. gitlab # [ 14.203446] systemd[1]: Reached target Network is Online. gitlab # [ 15.383102] gitaly[892]: time="2026-09-23T19:17:46.178Z" level=info msg="License database preloaded" duration_ms=3773 pid=892 gitlab # [ 58.501878] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[925]: W, [2026-09-23T19:18:29.297385 #925] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 59.499981] gitlab-workhorse[819]: time="2026-09-23T19:18:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 61.505804] postgres[1446]: [1446] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 61.506341] postgres[1446]: [1446] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 61.506474] postgres[1446]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 61.506599] postgres[1446]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 61.506721] postgres[1446]: attidentity AS identity, gitlab # [ 61.506842] postgres[1446]: attgenerated as attgenerated gitlab # [ 61.506963] postgres[1446]: FROM pg_attribute a gitlab # [ 61.507091] postgres[1446]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 61.507216] postgres[1446]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 61.507387] postgres[1446]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 61.507510] postgres[1446]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 61.507635] postgres[1446]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 61.507757] postgres[1446]: ORDER BY a.attnum gitlab # [ 61.507877] postgres[1446]: gitlab # [ 66.621080] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1488]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 66.627255] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1488]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 69.506717] gitlab-workhorse[819]: time="2026-09-23T19:18:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.518525] gitlab-workhorse[819]: time="2026-09-23T19:18:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 89.525454] gitlab-workhorse[819]: time="2026-09-23T19:19:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 93.644246] postgres[1600]: [1600] ERROR: relation "table_without_models" does not exist gitlab # [ 93.644643] postgres[1600]: [1600] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 99.532443] gitlab-workhorse[819]: time="2026-09-23T19:19:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.533940] gitlab-workhorse[819]: time="2026-09-23T19:19:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 119.544682] gitlab-workhorse[819]: time="2026-09-23T19:19:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 123.735008] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 123.739462] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 123.743350] systemd[1]: Starting gitlab.service... gitlab # [ 124.335414] bundle[1975]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 124.341882] bundle[1975]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 124.347786] bundle[1975]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 128.939644] sidekiq-cluster[1974]: {"severity":"INFO","time":"2026-09-23T19:19:39.694Z","message":"Starting cluster with 1 processes"} gitlab # [ 128.998064] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.763Z","pid":1975,"message":"* Enabling systemd notification integration"} gitlab # [ 129.003080] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"Puma starting in cluster mode..."} gitlab # [ 129.007555] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 129.012349] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 129.018582] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Min threads: 0"} gitlab # [ 129.022608] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Max threads: 4"} gitlab # [ 129.026568] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Environment: production"} gitlab # [ 129.030887] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Master PID: 1975"} gitlab # [ 129.034880] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Workers: 2"} gitlab # [ 129.038751] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 129.043699] bundle[1975]: {"timestamp":"2026-09-23T19:19:39.794Z","pid":1975,"message":"* Preloading application"} gitlab # [ 129.550163] gitlab-workhorse[819]: time="2026-09-23T19:19:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 139.551403] gitlab-workhorse[819]: time="2026-09-23T19:19:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 149.562281] gitlab-workhorse[819]: time="2026-09-23T19:20:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 159.573026] gitlab-workhorse[819]: time="2026-09-23T19:20:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 166.954557] bundle[1975]: W, [2026-09-23T19:20:17.750231 #1975] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 169.583258] gitlab-workhorse[819]: time="2026-09-23T19:20:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 170.910232] sidekiq-cluster[2024]: 2026-09-23T19:20:21.705Z pid=2024 tid=wn4 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2024, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432082944} gitlab # [ 171.215878] sidekiq-cluster[2024]: W, [2026-09-23T19:20:22.012554 #2024] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 171.283439] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:22.080Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 175.146041] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:25.941Z","message":"GitLab reliable fetch activated!"} gitlab # [ 175.146217] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:25.941Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 175.218451] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.015Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 175.225575] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.020Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 175.232615] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.026Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 175.239170] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.031Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 175.246494] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.036Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 175.252339] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.040Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 175.257587] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.043Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 175.262283] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.046Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 175.266698] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.049Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 175.270886] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.052Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 175.275285] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.055Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 175.279300] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.058Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 175.282814] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.061Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 175.286420] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.064Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 175.289787] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.067Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 175.293006] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.069Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 175.295876] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.073Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 175.298789] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.076Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 175.301828] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.080Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 175.304535] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.083Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 175.307439] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.086Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 175.310540] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.088Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 175.313123] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.091Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 175.315608] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.094Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 175.318302] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.096Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 175.320678] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.099Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 175.322871] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.103Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 175.326417] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.106Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 175.328784] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.109Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 175.331454] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.112Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 175.334120] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.115Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 175.337009] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.118Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 175.339505] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.121Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 175.342371] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.124Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 175.345098] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.126Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 175.347753] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.128Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 175.350612] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.130Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 175.353245] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.133Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 175.355794] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.135Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 175.358464] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.137Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 175.361333] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.139Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 175.363970] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.141Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 175.366844] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.143Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 175.369440] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.145Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 175.371973] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.147Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 175.374755] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.150Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 175.377294] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.151Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 175.379668] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.154Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 175.382164] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.156Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 175.384529] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.158Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 175.386825] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.161Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 175.389179] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.163Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 175.391381] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.176Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 175.393959] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.179Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 175.396240] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.182Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 175.398417] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.184Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 175.400610] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.187Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 175.402877] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.190Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 175.405176] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.191Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 175.407375] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.194Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 175.409478] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.195Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 175.411441] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.197Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 175.413653] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.200Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 175.415756] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.203Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 175.418108] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.206Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 175.420261] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.209Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 175.422281] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.211Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 175.424449] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.213Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 175.426697] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.215Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 175.429052] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.216Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 175.431250] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.218Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 175.433570] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.220Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 175.435759] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.222Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 175.438207] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.223Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 175.440452] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.225Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 175.443131] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.227Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 175.445469] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.229Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 175.447854] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.231Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 175.450172] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.232Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 175.452286] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.234Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 175.454579] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.236Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 175.457046] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.238Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 175.459306] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.240Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 175.461852] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.242Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 175.464191] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.244Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 175.466338] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.246Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 175.468679] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.249Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 175.470918] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.251Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 175.473296] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.253Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 175.475519] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.255Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 175.477940] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.257Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 175.480138] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.259Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 175.482372] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.261Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 175.484799] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.263Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 175.486780] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.266Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 175.489191] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.269Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 175.491417] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.271Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 175.493750] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.273Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 175.495970] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.286Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 175.498306] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.288Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 175.500468] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.291Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 175.502688] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.294Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 175.504755] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.298Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 175.506903] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.301Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 175.557663] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.352Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 175.561019] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.357Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 175.564284] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.359Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 175.567743] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.362Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 175.571308] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:20:26.366Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 179.590164] gitlab-workhorse[819]: time="2026-09-23T19:20:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 189.601135] gitlab-workhorse[819]: time="2026-09-23T19:20:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 199.602227] gitlab-workhorse[819]: time="2026-09-23T19:20:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 209.611426] gitlab-workhorse[819]: time="2026-09-23T19:21:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 216.000401] postgres[2752]: [2752] ERROR: relation "table_without_models" does not exist gitlab # [ 216.000826] postgres[2752]: [2752] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 219.434843] postgres[2773]: [2773] ERROR: relation "table_without_models" does not exist gitlab # [ 219.435239] postgres[2773]: [2773] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 219.612673] gitlab-workhorse[819]: time="2026-09-23T19:21:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 222.420941] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:13.210Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 222.423434] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:13.220Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 222.431093] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:13.220Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 222.436658] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:13.220Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 222.443326] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:13.228Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 225.176310] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:15.971Z","message":"Cleaning working queues"} gitlab # [ 225.185011] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:15.981Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"292f8da1fd8846c3bb3ed400","created_at":"2026-09-23T19:19:33.998Z","trace_propagation_headers":{"sentry-trace":"83406048dd134cbcb6b8687185746c8c-26bf92289f60457d","baggage":"sentry-trace_id=83406048dd134cbcb6b8687185746c8c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"f63e83f0d2612dcbe010855000b6ba2b","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:34.010Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-292f8da1fd8846c3bb3ed400: start","job_status":"start","queue_duration_s":101.970705,"scheduling_latency_s":101.970808} gitlab # [ 225.612572] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:16.408Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"292f8da1fd8846c3bb3ed400","created_at":"2026-09-23T19:19:33.998Z","trace_propagation_headers":{"sentry-trace":"83406048dd134cbcb6b8687185746c8c-26bf92289f60457d","baggage":"sentry-trace_id=83406048dd134cbcb6b8687185746c8c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"f63e83f0d2612dcbe010855000b6ba2b","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:34.010Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-292f8da1fd8846c3bb3ed400: done: 0.427619 sec","job_status":"done","queue_duration_s":101.970705,"scheduling_latency_s":101.970808,"redis_calls":29,"redis_duration_s":0.016087,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.010611,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003077,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002399,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.192645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19845082,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.427619,"completed_at":"2026-09-23T19:21:16.408Z","load_balancing_strategy":"primary","db_duration_s":0.108703,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 226.837123] bundle[1975]: {"timestamp":"2026-09-23T19:21:17.632Z","pid":1975,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 226.837449] bundle[1975]: {"timestamp":"2026-09-23T19:21:17.632Z","pid":1975,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 226.837618] bundle[1975]: {"timestamp":"2026-09-23T19:21:17.633Z","pid":1975,"message":"! #\u003cThread:0x0000fdebd3fdb070 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 226.837753] bundle[1975]: {"timestamp":"2026-09-23T19:21:17.633Z","pid":1975,"message":"! #\u003cThread:0x0000fdebd41d31e8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 226.837859] bundle[1975]: {"timestamp":"2026-09-23T19:21:17.633Z","pid":1975,"message":"Use Ctrl-C to stop"} gitlab # [ 226.838015] bundle[1975]: {"timestamp":"2026-09-23T19:21:17.633Z","pid":1975,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 230.086911] bundle[1975]: {"timestamp":"2026-09-23T19:21:20.883Z","pid":1975,"message":"- Worker 0 (PID: 2909) booted in 3.21s, phase: 0"} gitlab # [ 230.088266] bundle[1975]: {"timestamp":"2026-09-23T19:21:20.884Z","pid":1975,"message":"- Worker 1 (PID: 2913) booted in 3.19s, phase: 0"} gitlab # [ 230.090941] systemd[1]: Started gitlab.service. gitlab # [ 230.093300] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 230.100462] systemd[1]: Starting setup-gitlab.service... gitlab # [ 230.136706] setup-gitlab-start[2966]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 231.108212] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 926 gitlab # [ 231.112541] setup-gitlab-start[2966]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 231.117402] setup-gitlab-start[2966]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 231.133900] runuser[2983]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 231.698452] setup-gitlab-start[2984]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.698849] setup-gitlab-start[2984]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.700604] setup-gitlab-start[2984]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 232.549287] setup-gitlab-start[2984]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 232.549696] setup-gitlab-start[2984]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 232.552086] setup-gitlab-start[2984]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 238.397678] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:29.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f146b1ec2a7324f6395024b9","created_at":"2026-09-23T19:21:29.078Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.189Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f146b1ec2a7324f6395024b9: start","job_status":"start","queue_duration_s":0.003593,"scheduling_latency_s":0.003656} gitlab # [ 239.344162] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.139Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f146b1ec2a7324f6395024b9","created_at":"2026-09-23T19:21:29.078Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.189Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f146b1ec2a7324f6395024b9: done: 0.947158 sec","job_status":"done","queue_duration_s":0.003593,"scheduling_latency_s":0.003656,"redis_calls":71,"redis_duration_s":0.325627,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.039719,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.004459,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.274429,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00702,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.314684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00190842,"duration_s":0.947158,"completed_at":"2026-09-23T19:21:30.139Z","load_balancing_strategy":"primary","db_duration_s":0.064841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.440663] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.152Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8421af9c038e2981c10b7faa","created_at":"2026-09-23T19:21:29.206Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b163d003fd6d93bd0052c34eb87d22d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.223Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8421af9c038e2981c10b7faa: start","job_status":"start","queue_duration_s":0.929148,"scheduling_latency_s":0.929202} gitlab # [ 239.532307] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.327Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8421af9c038e2981c10b7faa","created_at":"2026-09-23T19:21:29.206Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b163d003fd6d93bd0052c34eb87d22d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.223Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8421af9c038e2981c10b7faa: done: 0.174809 sec","job_status":"done","queue_duration_s":0.929148,"scheduling_latency_s":0.929202,"redis_calls":13,"redis_duration_s":0.059891,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003577,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.051016,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005298,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.02590424,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.174809,"completed_at":"2026-09-23T19:21:30.327Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01506,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 239.628248] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8959ed27d2eac70231107fd1","created_at":"2026-09-23T19:21:29.230Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b12df5bb4694b3adca5b335cad29871","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.242Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8959ed27d2eac70231107fd1: start","job_status":"start","queue_duration_s":1.161819,"scheduling_latency_s":1.161879} gitlab # [ 239.785322] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8959ed27d2eac70231107fd1","created_at":"2026-09-23T19:21:29.230Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b12df5bb4694b3adca5b335cad29871","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.242Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8959ed27d2eac70231107fd1: done: 0.176603 sec","job_status":"done","queue_duration_s":1.161819,"scheduling_latency_s":1.161879,"redis_calls":14,"redis_duration_s":0.007207,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002434,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002151,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00142672,"exclusive_lock_hold_duration_s":0.10706036,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.176603,"completed_at":"2026-09-23T19:21:30.580Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.890005] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.584Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"15da529649f0c1c378d67bc1","created_at":"2026-09-23T19:21:29.271Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0517833cb0f880f4f5abe42c8178a8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.291Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-15da529649f0c1c378d67bc1: start","job_status":"start","queue_duration_s":1.292936,"scheduling_latency_s":1.292982} gitlab # [ 239.908587] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.609Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 239.912619] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"15da529649f0c1c378d67bc1","created_at":"2026-09-23T19:21:29.271Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0517833cb0f880f4f5abe42c8178a8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.291Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-15da529649f0c1c378d67bc1: done: 0.039362 sec","job_status":"done","queue_duration_s":1.292936,"scheduling_latency_s":1.292982,"redis_calls":10,"redis_duration_s":0.0035640000000000003,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000935,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000674,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050424,"duration_s":0.039362,"completed_at":"2026-09-23T19:21:30.623Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.956265] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.626Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e00115414d1f1e665bf3e692","created_at":"2026-09-23T19:21:29.304Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"579675c4c392e913142a7a34ec2edbda","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.319Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e00115414d1f1e665bf3e692: start","job_status":"start","queue_duration_s":1.307085,"scheduling_latency_s":1.307116} gitlab # [ 239.970683] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.645Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 239.974526] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e00115414d1f1e665bf3e692","created_at":"2026-09-23T19:21:29.304Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"579675c4c392e913142a7a34ec2edbda","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.319Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e00115414d1f1e665bf3e692: done: 0.030334 sec","job_status":"done","queue_duration_s":1.307085,"scheduling_latency_s":1.307116,"redis_calls":10,"redis_duration_s":0.0027600000000000003,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000591,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044544,"duration_s":0.030334,"completed_at":"2026-09-23T19:21:30.656Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.016801] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"629046fa5342d903b11f8375","created_at":"2026-09-23T19:21:29.335Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c17e23de830c5bae6bbb82e09ce76cdb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:21:29.345Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-629046fa5342d903b11f8375: start","job_status":"start","queue_duration_s":1.312871,"scheduling_latency_s":1.312895} gitlab # [ 240.031367] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.675Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 240.034776] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.684Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"629046fa5342d903b11f8375","created_at":"2026-09-23T19:21:29.335Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c17e23de830c5bae6bbb82e09ce76cdb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:21:29.345Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-629046fa5342d903b11f8375: done: 0.025622 sec","job_status":"done","queue_duration_s":1.312871,"scheduling_latency_s":1.312895,"redis_calls":10,"redis_duration_s":0.002957,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000552,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001512,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042348,"duration_s":0.025622,"completed_at":"2026-09-23T19:21:30.684Z","load_balancing_strategy":"primary","db_duration_s":0.000832,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.076035] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.686Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"79020e3c373f7d5d337279a1","created_at":"2026-09-23T19:21:29.376Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a8abfc4bc2d38d3d5e5e4b2f8c2fe9f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.399Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-79020e3c373f7d5d337279a1: start","job_status":"start","queue_duration_s":1.286712,"scheduling_latency_s":1.286735} gitlab # [ 240.090616] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"79020e3c373f7d5d337279a1","created_at":"2026-09-23T19:21:29.376Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a8abfc4bc2d38d3d5e5e4b2f8c2fe9f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.399Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-79020e3c373f7d5d337279a1: done: 0.028873 sec","job_status":"done","queue_duration_s":1.286712,"scheduling_latency_s":1.286735,"redis_calls":12,"redis_duration_s":0.003199,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001212,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001472,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003661,"duration_s":0.028873,"completed_at":"2026-09-23T19:21:30.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0015,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.132747] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.716Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7360dac1e7e0557654f221f1","created_at":"2026-09-23T19:21:29.437Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99f91a3f01f91318043727b921f34a0a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.472Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7360dac1e7e0557654f221f1: start","job_status":"start","queue_duration_s":1.244696,"scheduling_latency_s":1.244717} gitlab # [ 240.147145] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.739Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7360dac1e7e0557654f221f1","created_at":"2026-09-23T19:21:29.437Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99f91a3f01f91318043727b921f34a0a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.472Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7360dac1e7e0557654f221f1: done: 0.022323 sec","job_status":"done","queue_duration_s":1.244696,"scheduling_latency_s":1.244717,"redis_calls":10,"redis_duration_s":0.002379,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000487,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001387,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033204,"duration_s":0.022323,"completed_at":"2026-09-23T19:21:30.739Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.189566] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:30.740Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"8f152d77ec05e3233bbeb843","created_at":"2026-09-23T19:21:29.520Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdee0d89c20e39bc76bb4fa18ae263d6","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.546Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-8f152d77ec05e3233bbeb843: start","job_status":"start","queue_duration_s":1.193892,"scheduling_latency_s":1.193909} gitlab # [ 240.363838] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.159Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"8f152d77ec05e3233bbeb843","created_at":"2026-09-23T19:21:29.520Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdee0d89c20e39bc76bb4fa18ae263d6","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.546Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-8f152d77ec05e3233bbeb843: done: 0.419198 sec","job_status":"done","queue_duration_s":1.193892,"scheduling_latency_s":1.193909,"redis_calls":565,"redis_duration_s":0.092766,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000486,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.091083,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.367448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029898,"duration_s":0.419198,"completed_at":"2026-09-23T19:21:31.159Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.421853] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.162Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"d6594efede762420a2bbb370","created_at":"2026-09-23T19:21:29.571Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7de161fd067da3629f700d5418e12960","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.596Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-d6594efede762420a2bbb370: start","job_status":"start","queue_duration_s":1.565719,"scheduling_latency_s":1.565742} gitlab # [ 240.632615] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.427Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"d6594efede762420a2bbb370","created_at":"2026-09-23T19:21:29.571Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7de161fd067da3629f700d5418e12960","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.596Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-d6594efede762420a2bbb370: done: 0.265561 sec","job_status":"done","queue_duration_s":1.565719,"scheduling_latency_s":1.565742,"redis_calls":12,"redis_duration_s":0.003351,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000565,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.05,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.19382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00062102,"exclusive_lock_hold_duration_s":0.24101322,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.265561,"completed_at":"2026-09-23T19:21:31.427Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.078321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.734076] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d38fb284a17b8c32581e6876","created_at":"2026-09-23T19:21:29.678Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b4c116e868543bef73747c84ca761a0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.699Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d38fb284a17b8c32581e6876: start","job_status":"start","queue_duration_s":1.73,"scheduling_latency_s":1.730019} gitlab # [ 240.751251] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.484Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d38fb284a17b8c32581e6876","created_at":"2026-09-23T19:21:29.678Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b4c116e868543bef73747c84ca761a0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.699Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d38fb284a17b8c32581e6876: done: 0.054375 sec","job_status":"done","queue_duration_s":1.73,"scheduling_latency_s":1.730019,"redis_calls":10,"redis_duration_s":0.003078,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000586,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001904,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055334,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.054375,"completed_at":"2026-09-23T19:21:31.483Z","load_balancing_strategy":"primary","db_duration_s":0.027664,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.795592] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.486Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"618fe60629e56520e2abe3fc","created_at":"2026-09-23T19:21:29.724Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1fa0a5534502c88bf9619a2c42371c30","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.757Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-618fe60629e56520e2abe3fc: start","job_status":"start","queue_duration_s":1.728921,"scheduling_latency_s":1.728944} gitlab # [ 240.809199] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"618fe60629e56520e2abe3fc","created_at":"2026-09-23T19:21:29.724Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1fa0a5534502c88bf9619a2c42371c30","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.757Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-618fe60629e56520e2abe3fc: done: 0.033821 sec","job_status":"done","queue_duration_s":1.728921,"scheduling_latency_s":1.728944,"redis_calls":12,"redis_duration_s":0.00318,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001441,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006089,"exclusive_lock_hold_duration_s":0.010385100000000001,"duration_s":0.033821,"completed_at":"2026-09-23T19:21:31.519Z","load_balancing_strategy":"primary","db_duration_s":0.006345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.851527] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a2b84371a1f0765856eed77b","created_at":"2026-09-23T19:21:29.784Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3aab247b94a63c90ad8d6958707a966","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.807Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a2b84371a1f0765856eed77b: start","job_status":"start","queue_duration_s":1.714546,"scheduling_latency_s":1.714566} gitlab # [ 240.865998] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a2b84371a1f0765856eed77b","created_at":"2026-09-23T19:21:29.784Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3aab247b94a63c90ad8d6958707a966","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.807Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a2b84371a1f0765856eed77b: done: 0.026488 sec","job_status":"done","queue_duration_s":1.714546,"scheduling_latency_s":1.714566,"redis_calls":12,"redis_duration_s":0.003181,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001506,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001134,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00056946,"exclusive_lock_hold_duration_s":0.00454078,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.026488,"completed_at":"2026-09-23T19:21:31.548Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.908970] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5883267b78af982e9cd3b18c","created_at":"2026-09-23T19:21:29.861Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.892Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5883267b78af982e9cd3b18c: start","job_status":"start","queue_duration_s":1.657592,"scheduling_latency_s":1.65761} gitlab # [ 240.922389] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5883267b78af982e9cd3b18c","created_at":"2026-09-23T19:21:29.861Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:29.892Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5883267b78af982e9cd3b18c: done: 0.040016 sec","job_status":"done","queue_duration_s":1.657592,"scheduling_latency_s":1.65761,"redis_calls":19,"redis_duration_s":0.004934,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001114,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000381,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002651,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032058,"duration_s":0.040016,"completed_at":"2026-09-23T19:21:31.589Z","load_balancing_strategy":"primary","db_duration_s":0.003527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.964758] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f83ed7c53ae488bfa93af79b","created_at":"2026-09-23T19:21:29.921Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21f35f9b7b0cffebe489f3b6f8057472","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:21:29.947Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f83ed7c53ae488bfa93af79b: start","job_status":"start","queue_duration_s":1.643996,"scheduling_latency_s":1.644012} gitlab # [ 240.977737] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.603Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 240.981485] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.611Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f83ed7c53ae488bfa93af79b","created_at":"2026-09-23T19:21:29.921Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21f35f9b7b0cffebe489f3b6f8057472","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:21:29.947Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f83ed7c53ae488bfa93af79b: done: 0.019847 sec","job_status":"done","queue_duration_s":1.643996,"scheduling_latency_s":1.644012,"redis_calls":10,"redis_duration_s":0.0023680000000000003,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000497,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032036,"duration_s":0.019847,"completed_at":"2026-09-23T19:21:31.611Z","load_balancing_strategy":"primary","db_duration_s":0.000695,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.022867] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.612Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0d26693ebbb2bc44d3478755","created_at":"2026-09-23T19:21:29.974Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"353fdb26cefd17ffe217c1a6e22eacb6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.001Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0d26693ebbb2bc44d3478755: start","job_status":"start","queue_duration_s":1.610813,"scheduling_latency_s":1.610832} gitlab # [ 241.037720] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.624Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 241.041112] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0d26693ebbb2bc44d3478755","created_at":"2026-09-23T19:21:29.974Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"353fdb26cefd17ffe217c1a6e22eacb6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.001Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0d26693ebbb2bc44d3478755: done: 0.020345 sec","job_status":"done","queue_duration_s":1.610813,"scheduling_latency_s":1.610832,"redis_calls":10,"redis_duration_s":0.002393,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000532,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033468,"duration_s":0.020345,"completed_at":"2026-09-23T19:21:31.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.084043] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b7e77f8389192158038ebe1","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b7e77f8389192158038ebe1: start","job_status":"start","queue_duration_s":1.553267,"scheduling_latency_s":1.553283} gitlab # [ 241.096226] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b7e77f8389192158038ebe1","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b7e77f8389192158038ebe1: done: 0.023942 sec","job_status":"done","queue_duration_s":1.553267,"scheduling_latency_s":1.553283,"redis_calls":16,"redis_duration_s":0.004289,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002728,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030942,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023942,"completed_at":"2026-09-23T19:21:31.658Z","load_balancing_strategy":"primary","db_duration_s":0.00059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.136676] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbbcd1d8dd04f0abfa30e222","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbbcd1d8dd04f0abfa30e222: start","job_status":"start","queue_duration_s":1.578615,"scheduling_latency_s":1.57863} gitlab # [ 241.149024] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbbcd1d8dd04f0abfa30e222","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbbcd1d8dd04f0abfa30e222: done: 0.016744 sec","job_status":"done","queue_duration_s":1.578615,"scheduling_latency_s":1.57863,"redis_calls":13,"redis_duration_s":0.002805,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001844,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016744,"completed_at":"2026-09-23T19:21:31.676Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.188106] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d510e863ac3d3059a7ce8e2e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d510e863ac3d3059a7ce8e2e: start","job_status":"start","queue_duration_s":1.596734,"scheduling_latency_s":1.596751} gitlab # [ 241.200100] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d510e863ac3d3059a7ce8e2e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d510e863ac3d3059a7ce8e2e: done: 0.016385 sec","job_status":"done","queue_duration_s":1.596734,"scheduling_latency_s":1.596751,"redis_calls":13,"redis_duration_s":0.002569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001633,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016385,"completed_at":"2026-09-23T19:21:31.694Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.239167] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"619e39748962b61b4bc11bc0","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-619e39748962b61b4bc11bc0: start","job_status":"start","queue_duration_s":1.614722,"scheduling_latency_s":1.614739} gitlab # [ 241.251800] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"619e39748962b61b4bc11bc0","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-619e39748962b61b4bc11bc0: done: 0.016214 sec","job_status":"done","queue_duration_s":1.614722,"scheduling_latency_s":1.614739,"redis_calls":13,"redis_duration_s":0.002744,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016214,"completed_at":"2026-09-23T19:21:31.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.292271] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f021c44e1f21d2a7ebc1809","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f021c44e1f21d2a7ebc1809: start","job_status":"start","queue_duration_s":1.632348,"scheduling_latency_s":1.632366} gitlab # [ 241.304863] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f021c44e1f21d2a7ebc1809","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f021c44e1f21d2a7ebc1809: done: 0.016096 sec","job_status":"done","queue_duration_s":1.632348,"scheduling_latency_s":1.632366,"redis_calls":13,"redis_duration_s":0.0027559999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001844,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016096,"completed_at":"2026-09-23T19:21:31.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.344281] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"757d49b1567dc77110c802aa","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-757d49b1567dc77110c802aa: start","job_status":"start","queue_duration_s":1.649966,"scheduling_latency_s":1.649982} gitlab # [ 241.356442] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"757d49b1567dc77110c802aa","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-757d49b1567dc77110c802aa: done: 0.016048 sec","job_status":"done","queue_duration_s":1.649966,"scheduling_latency_s":1.649982,"redis_calls":13,"redis_duration_s":0.002686,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016048,"completed_at":"2026-09-23T19:21:31.747Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.395755] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4758cdec1bdb93a40fce2367","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4758cdec1bdb93a40fce2367: start","job_status":"start","queue_duration_s":1.667716,"scheduling_latency_s":1.667733} gitlab # [ 241.408495] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4758cdec1bdb93a40fce2367","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4758cdec1bdb93a40fce2367: done: 0.016563 sec","job_status":"done","queue_duration_s":1.667716,"scheduling_latency_s":1.667733,"redis_calls":13,"redis_duration_s":0.002722,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016563,"completed_at":"2026-09-23T19:21:31.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.447788] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21166728be49fa202f505a7e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21166728be49fa202f505a7e: start","job_status":"start","queue_duration_s":1.685833,"scheduling_latency_s":1.68585} gitlab # [ 241.461800] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21166728be49fa202f505a7e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21166728be49fa202f505a7e: done: 0.016329 sec","job_status":"done","queue_duration_s":1.685833,"scheduling_latency_s":1.68585,"redis_calls":13,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001036,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016329,"completed_at":"2026-09-23T19:21:31.783Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.501466] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55858ab57a0ca7f835aef2aa","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55858ab57a0ca7f835aef2aa: start","job_status":"start","queue_duration_s":1.703741,"scheduling_latency_s":1.703758} gitlab # [ 241.513965] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55858ab57a0ca7f835aef2aa","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55858ab57a0ca7f835aef2aa: done: 0.015917 sec","job_status":"done","queue_duration_s":1.703741,"scheduling_latency_s":1.703758,"redis_calls":13,"redis_duration_s":0.002762,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015917,"completed_at":"2026-09-23T19:21:31.800Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.553113] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"227af8f6482add097a814883","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-227af8f6482add097a814883: start","job_status":"start","queue_duration_s":1.720837,"scheduling_latency_s":1.720852} gitlab # [ 241.565545] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"227af8f6482add097a814883","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-227af8f6482add097a814883: done: 0.016227 sec","job_status":"done","queue_duration_s":1.720837,"scheduling_latency_s":1.720852,"redis_calls":13,"redis_duration_s":0.002705,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001793,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016227,"completed_at":"2026-09-23T19:21:31.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.604456] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51e4b056bdfc3862df9bf615","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51e4b056bdfc3862df9bf615: start","job_status":"start","queue_duration_s":1.738395,"scheduling_latency_s":1.738411} gitlab # [ 241.616597] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51e4b056bdfc3862df9bf615","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51e4b056bdfc3862df9bf615: done: 0.016035 sec","job_status":"done","queue_duration_s":1.738395,"scheduling_latency_s":1.738411,"redis_calls":13,"redis_duration_s":0.002969,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016035,"completed_at":"2026-09-23T19:21:31.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.655973] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"831aff50fc3989a4f98964fb","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-831aff50fc3989a4f98964fb: start","job_status":"start","queue_duration_s":1.755761,"scheduling_latency_s":1.755777} gitlab # [ 241.670036] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"831aff50fc3989a4f98964fb","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-831aff50fc3989a4f98964fb: done: 0.016324 sec","job_status":"done","queue_duration_s":1.755761,"scheduling_latency_s":1.755777,"redis_calls":13,"redis_duration_s":0.0029310000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016324,"completed_at":"2026-09-23T19:21:31.853Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.709075] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2090892be60cff43d56357c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2090892be60cff43d56357c: start","job_status":"start","queue_duration_s":1.773498,"scheduling_latency_s":1.773513} gitlab # [ 241.721343] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2090892be60cff43d56357c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2090892be60cff43d56357c: done: 0.016675 sec","job_status":"done","queue_duration_s":1.773498,"scheduling_latency_s":1.773513,"redis_calls":13,"redis_duration_s":0.002662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001691,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016675,"completed_at":"2026-09-23T19:21:31.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.759916] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae06e7e9745137a32c18883e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae06e7e9745137a32c18883e: start","job_status":"start","queue_duration_s":1.791581,"scheduling_latency_s":1.791598} gitlab # [ 241.772150] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae06e7e9745137a32c18883e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae06e7e9745137a32c18883e: done: 0.015772 sec","job_status":"done","queue_duration_s":1.791581,"scheduling_latency_s":1.791598,"redis_calls":13,"redis_duration_s":0.002744,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001114,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015772,"completed_at":"2026-09-23T19:21:31.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.810362] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feb96522bbf651dfd8c5d876","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feb96522bbf651dfd8c5d876: start","job_status":"start","queue_duration_s":1.808824,"scheduling_latency_s":1.808841} gitlab # [ 241.822540] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feb96522bbf651dfd8c5d876","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feb96522bbf651dfd8c5d876: done: 0.015957 sec","job_status":"done","queue_duration_s":1.808824,"scheduling_latency_s":1.808841,"redis_calls":13,"redis_duration_s":0.002764,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001814,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015957,"completed_at":"2026-09-23T19:21:31.905Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.862337] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"747eca95da533618acdb644b","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-747eca95da533618acdb644b: start","job_status":"start","queue_duration_s":1.826286,"scheduling_latency_s":1.826302} gitlab # [ 241.875861] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"747eca95da533618acdb644b","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-747eca95da533618acdb644b: done: 0.015855 sec","job_status":"done","queue_duration_s":1.826286,"scheduling_latency_s":1.826302,"redis_calls":13,"redis_duration_s":0.002765,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001797,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015855,"completed_at":"2026-09-23T19:21:31.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.914359] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff063ea61c3ebcac1f412902","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff063ea61c3ebcac1f412902: start","job_status":"start","queue_duration_s":1.843785,"scheduling_latency_s":1.843801} gitlab # [ 241.926699] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff063ea61c3ebcac1f412902","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff063ea61c3ebcac1f412902: done: 0.016074 sec","job_status":"done","queue_duration_s":1.843785,"scheduling_latency_s":1.843801,"redis_calls":13,"redis_duration_s":0.002659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016074,"completed_at":"2026-09-23T19:21:31.940Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.964879] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afd8a3cd909772059ed9071e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afd8a3cd909772059ed9071e: start","job_status":"start","queue_duration_s":1.861536,"scheduling_latency_s":1.861554} gitlab # [ 241.977110] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afd8a3cd909772059ed9071e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afd8a3cd909772059ed9071e: done: 0.01579 sec","job_status":"done","queue_duration_s":1.861536,"scheduling_latency_s":1.861554,"redis_calls":13,"redis_duration_s":0.002658,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001752,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01579,"completed_at":"2026-09-23T19:21:31.958Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.015662] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f80da2330b1bae3c6f97004d","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f80da2330b1bae3c6f97004d: start","job_status":"start","queue_duration_s":1.87881,"scheduling_latency_s":1.878826} gitlab # [ 242.028073] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f80da2330b1bae3c6f97004d","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f80da2330b1bae3c6f97004d: done: 0.01569 sec","job_status":"done","queue_duration_s":1.87881,"scheduling_latency_s":1.878826,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01569,"completed_at":"2026-09-23T19:21:31.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.066769] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15ad0ad6ed1338b6e22efa96","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15ad0ad6ed1338b6e22efa96: start","job_status":"start","queue_duration_s":1.896021,"scheduling_latency_s":1.896038} gitlab # [ 242.080243] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15ad0ad6ed1338b6e22efa96","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15ad0ad6ed1338b6e22efa96: done: 0.016455 sec","job_status":"done","queue_duration_s":1.896021,"scheduling_latency_s":1.896038,"redis_calls":13,"redis_duration_s":0.003062,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016455,"completed_at":"2026-09-23T19:21:31.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.119952] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:31.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a6cbf92146b5605537fb568","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a6cbf92146b5605537fb568: start","job_status":"start","queue_duration_s":1.913807,"scheduling_latency_s":1.913824} gitlab # [ 242.132313] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a6cbf92146b5605537fb568","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a6cbf92146b5605537fb568: done: 0.016349 sec","job_status":"done","queue_duration_s":1.913807,"scheduling_latency_s":1.913824,"redis_calls":13,"redis_duration_s":0.0028659999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016349,"completed_at":"2026-09-23T19:21:32.011Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.171964] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16958a1e89d82589b563541c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16958a1e89d82589b563541c: start","job_status":"start","queue_duration_s":1.931464,"scheduling_latency_s":1.93148} gitlab # [ 242.184627] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16958a1e89d82589b563541c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16958a1e89d82589b563541c: done: 0.015884 sec","job_status":"done","queue_duration_s":1.931464,"scheduling_latency_s":1.93148,"redis_calls":13,"redis_duration_s":0.002979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015884,"completed_at":"2026-09-23T19:21:32.028Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.223458] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9b05ef1a49240817daeb9ca","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9b05ef1a49240817daeb9ca: start","job_status":"start","queue_duration_s":1.948619,"scheduling_latency_s":1.948635} gitlab # [ 242.235813] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9b05ef1a49240817daeb9ca","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9b05ef1a49240817daeb9ca: done: 0.016039 sec","job_status":"done","queue_duration_s":1.948619,"scheduling_latency_s":1.948635,"redis_calls":13,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016039,"completed_at":"2026-09-23T19:21:32.045Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.275162] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8324b7b1bf890b8559fc95e0","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8324b7b1bf890b8559fc95e0: start","job_status":"start","queue_duration_s":1.965888,"scheduling_latency_s":1.965905} gitlab # [ 242.287488] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8324b7b1bf890b8559fc95e0","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8324b7b1bf890b8559fc95e0: done: 0.016504 sec","job_status":"done","queue_duration_s":1.965888,"scheduling_latency_s":1.965905,"redis_calls":13,"redis_duration_s":0.002023,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016504,"completed_at":"2026-09-23T19:21:32.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.325735] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c17fe0e29ef45340252a3c7e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c17fe0e29ef45340252a3c7e: start","job_status":"start","queue_duration_s":1.983665,"scheduling_latency_s":1.983682} gitlab # [ 242.337625] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c17fe0e29ef45340252a3c7e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c17fe0e29ef45340252a3c7e: done: 0.016017 sec","job_status":"done","queue_duration_s":1.983665,"scheduling_latency_s":1.983682,"redis_calls":13,"redis_duration_s":0.001557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016017,"completed_at":"2026-09-23T19:21:32.080Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.375999] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5f1650bfc68f936002f4b23","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5f1650bfc68f936002f4b23: start","job_status":"start","queue_duration_s":2.000982,"scheduling_latency_s":2.000999} gitlab # [ 242.387905] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5f1650bfc68f936002f4b23","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5f1650bfc68f936002f4b23: done: 0.01555 sec","job_status":"done","queue_duration_s":2.000982,"scheduling_latency_s":2.000999,"redis_calls":13,"redis_duration_s":0.0014199999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01555,"completed_at":"2026-09-23T19:21:32.097Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.426469] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a0f3d859ad02ea067cb7aad","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a0f3d859ad02ea067cb7aad: start","job_status":"start","queue_duration_s":2.01779,"scheduling_latency_s":2.017807} gitlab # [ 242.438867] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a0f3d859ad02ea067cb7aad","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a0f3d859ad02ea067cb7aad: done: 0.014978 sec","job_status":"done","queue_duration_s":2.01779,"scheduling_latency_s":2.017807,"redis_calls":13,"redis_duration_s":0.001422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014978,"completed_at":"2026-09-23T19:21:32.113Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.478426] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abe4842778ad269c1fa22e29","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abe4842778ad269c1fa22e29: start","job_status":"start","queue_duration_s":2.034007,"scheduling_latency_s":2.034024} gitlab # [ 242.490790] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abe4842778ad269c1fa22e29","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abe4842778ad269c1fa22e29: done: 0.015303 sec","job_status":"done","queue_duration_s":2.034007,"scheduling_latency_s":2.034024,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015303,"completed_at":"2026-09-23T19:21:32.130Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.529111] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10c3203d9a51607965cd56f2","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10c3203d9a51607965cd56f2: start","job_status":"start","queue_duration_s":2.050541,"scheduling_latency_s":2.050559} gitlab # [ 242.541513] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10c3203d9a51607965cd56f2","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10c3203d9a51607965cd56f2: done: 0.014983 sec","job_status":"done","queue_duration_s":2.050541,"scheduling_latency_s":2.050559,"redis_calls":13,"redis_duration_s":0.001405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014983,"completed_at":"2026-09-23T19:21:32.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.580229] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"711bb884643e9d08d27bb3fd","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-711bb884643e9d08d27bb3fd: start","job_status":"start","queue_duration_s":2.066562,"scheduling_latency_s":2.066578} gitlab # [ 242.592100] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"711bb884643e9d08d27bb3fd","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-711bb884643e9d08d27bb3fd: done: 0.014957 sec","job_status":"done","queue_duration_s":2.066562,"scheduling_latency_s":2.066578,"redis_calls":13,"redis_duration_s":0.0014060000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014957,"completed_at":"2026-09-23T19:21:32.162Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.631056] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9efb9fe819fdc21287d69ed2","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9efb9fe819fdc21287d69ed2: start","job_status":"start","queue_duration_s":2.082592,"scheduling_latency_s":2.082608} gitlab # [ 242.643421] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9efb9fe819fdc21287d69ed2","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9efb9fe819fdc21287d69ed2: done: 0.015189 sec","job_status":"done","queue_duration_s":2.082592,"scheduling_latency_s":2.082608,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015189,"completed_at":"2026-09-23T19:21:32.178Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.684548] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7be50554657bf001d68a4692","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7be50554657bf001d68a4692: start","job_status":"start","queue_duration_s":2.098928,"scheduling_latency_s":2.098946} gitlab # [ 242.696885] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7be50554657bf001d68a4692","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7be50554657bf001d68a4692: done: 0.015657 sec","job_status":"done","queue_duration_s":2.098928,"scheduling_latency_s":2.098946,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015657,"completed_at":"2026-09-23T19:21:32.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.736039] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41fe8515300c71739bdddb6e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41fe8515300c71739bdddb6e: start","job_status":"start","queue_duration_s":2.115772,"scheduling_latency_s":2.11579} gitlab # [ 242.748442] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41fe8515300c71739bdddb6e","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41fe8515300c71739bdddb6e: done: 0.01498 sec","job_status":"done","queue_duration_s":2.115772,"scheduling_latency_s":2.11579,"redis_calls":13,"redis_duration_s":0.001431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-09-23T19:21:32.211Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.787530] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe772a18b9c0024f7905b1b5","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe772a18b9c0024f7905b1b5: start","job_status":"start","queue_duration_s":2.131864,"scheduling_latency_s":2.131898} gitlab # [ 242.800166] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe772a18b9c0024f7905b1b5","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe772a18b9c0024f7905b1b5: done: 0.014841 sec","job_status":"done","queue_duration_s":2.131864,"scheduling_latency_s":2.131898,"redis_calls":13,"redis_duration_s":0.0014290000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014841,"completed_at":"2026-09-23T19:21:32.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.839443] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1168a666fe18d4b3225caec7","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1168a666fe18d4b3225caec7: start","job_status":"start","queue_duration_s":2.147915,"scheduling_latency_s":2.147932} gitlab # [ 242.852079] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1168a666fe18d4b3225caec7","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1168a666fe18d4b3225caec7: done: 0.014859 sec","job_status":"done","queue_duration_s":2.147915,"scheduling_latency_s":2.147932,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014859,"completed_at":"2026-09-23T19:21:32.243Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.892274] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b3a382e06c060d07a5d3e2b","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b3a382e06c060d07a5d3e2b: start","job_status":"start","queue_duration_s":2.164008,"scheduling_latency_s":2.164025} gitlab # [ 242.904704] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b3a382e06c060d07a5d3e2b","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b3a382e06c060d07a5d3e2b: done: 0.015407 sec","job_status":"done","queue_duration_s":2.164008,"scheduling_latency_s":2.164025,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015407,"completed_at":"2026-09-23T19:21:32.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.943887] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f455989d3cd986ef7e6abf1f","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f455989d3cd986ef7e6abf1f: start","job_status":"start","queue_duration_s":2.180752,"scheduling_latency_s":2.180769} gitlab # [ 242.956293] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f455989d3cd986ef7e6abf1f","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f455989d3cd986ef7e6abf1f: done: 0.015058 sec","job_status":"done","queue_duration_s":2.180752,"scheduling_latency_s":2.180769,"redis_calls":13,"redis_duration_s":0.001422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015058,"completed_at":"2026-09-23T19:21:32.276Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.995784] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb39ffd675a97e8e64670e7d","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb39ffd675a97e8e64670e7d: start","job_status":"start","queue_duration_s":2.197133,"scheduling_latency_s":2.197151} gitlab # [ 243.008466] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb39ffd675a97e8e64670e7d","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb39ffd675a97e8e64670e7d: done: 0.015283 sec","job_status":"done","queue_duration_s":2.197133,"scheduling_latency_s":2.197151,"redis_calls":13,"redis_duration_s":0.0016449999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015283,"completed_at":"2026-09-23T19:21:32.293Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.047752] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b705607b4ca7b0b6ee48c2c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b705607b4ca7b0b6ee48c2c: start","job_status":"start","queue_duration_s":2.213451,"scheduling_latency_s":2.213468} gitlab # [ 243.060342] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b705607b4ca7b0b6ee48c2c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b705607b4ca7b0b6ee48c2c: done: 0.015195 sec","job_status":"done","queue_duration_s":2.213451,"scheduling_latency_s":2.213468,"redis_calls":13,"redis_duration_s":0.001396,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015195,"completed_at":"2026-09-23T19:21:32.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.100620] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edf8884488909fd1e7c3a175","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edf8884488909fd1e7c3a175: start","job_status":"start","queue_duration_s":2.229676,"scheduling_latency_s":2.229692} gitlab # [ 243.112822] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edf8884488909fd1e7c3a175","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edf8884488909fd1e7c3a175: done: 0.015145 sec","job_status":"done","queue_duration_s":2.229676,"scheduling_latency_s":2.229692,"redis_calls":13,"redis_duration_s":0.001451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015145,"completed_at":"2026-09-23T19:21:32.325Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.150864] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56a9d6795349370d044a0dd1","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56a9d6795349370d044a0dd1: start","job_status":"start","queue_duration_s":2.245854,"scheduling_latency_s":2.24587} gitlab # [ 243.163140] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56a9d6795349370d044a0dd1","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56a9d6795349370d044a0dd1: done: 0.015335 sec","job_status":"done","queue_duration_s":2.245854,"scheduling_latency_s":2.24587,"redis_calls":13,"redis_duration_s":0.0018110000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000733,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015335,"completed_at":"2026-09-23T19:21:32.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.201419] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee8c4c25321dbe3effcd4b55","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee8c4c25321dbe3effcd4b55: start","job_status":"start","queue_duration_s":2.262265,"scheduling_latency_s":2.262282} gitlab # [ 243.213821] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee8c4c25321dbe3effcd4b55","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee8c4c25321dbe3effcd4b55: done: 0.015086 sec","job_status":"done","queue_duration_s":2.262265,"scheduling_latency_s":2.262282,"redis_calls":13,"redis_duration_s":0.001387,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015086,"completed_at":"2026-09-23T19:21:32.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.252203] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5eaae3777664255d4aa24f8","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5eaae3777664255d4aa24f8: start","job_status":"start","queue_duration_s":2.278561,"scheduling_latency_s":2.278578} gitlab # [ 243.264173] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5eaae3777664255d4aa24f8","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5eaae3777664255d4aa24f8: done: 0.014824 sec","job_status":"done","queue_duration_s":2.278561,"scheduling_latency_s":2.278578,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014824,"completed_at":"2026-09-23T19:21:32.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.303719] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f6a02615b858d0cbff34afc","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f6a02615b858d0cbff34afc: start","job_status":"start","queue_duration_s":2.29457,"scheduling_latency_s":2.294587} gitlab # [ 243.315892] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f6a02615b858d0cbff34afc","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f6a02615b858d0cbff34afc: done: 0.015246 sec","job_status":"done","queue_duration_s":2.29457,"scheduling_latency_s":2.294587,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015246,"completed_at":"2026-09-23T19:21:32.390Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.354383] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4da5833d07b08ff7bd56a6c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4da5833d07b08ff7bd56a6c: start","job_status":"start","queue_duration_s":2.311048,"scheduling_latency_s":2.311064} gitlab # [ 243.366491] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4da5833d07b08ff7bd56a6c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4da5833d07b08ff7bd56a6c: done: 0.015075 sec","job_status":"done","queue_duration_s":2.311048,"scheduling_latency_s":2.311064,"redis_calls":13,"redis_duration_s":0.001422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015075,"completed_at":"2026-09-23T19:21:32.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.404930] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1689f66bf5cbb49326c63d1a","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1689f66bf5cbb49326c63d1a: start","job_status":"start","queue_duration_s":2.327544,"scheduling_latency_s":2.327563} gitlab # [ 243.416865] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1689f66bf5cbb49326c63d1a","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1689f66bf5cbb49326c63d1a: done: 0.015226 sec","job_status":"done","queue_duration_s":2.327544,"scheduling_latency_s":2.327563,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015226,"completed_at":"2026-09-23T19:21:32.423Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.454847] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3028e7b8dd04aa29db20f60c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3028e7b8dd04aa29db20f60c: start","job_status":"start","queue_duration_s":2.343901,"scheduling_latency_s":2.343919} gitlab # [ 243.467844] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3028e7b8dd04aa29db20f60c","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3028e7b8dd04aa29db20f60c: done: 0.015509 sec","job_status":"done","queue_duration_s":2.343901,"scheduling_latency_s":2.343919,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015509,"completed_at":"2026-09-23T19:21:32.440Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.508494] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a95b31147550cb9ccc1ea61","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a95b31147550cb9ccc1ea61: start","job_status":"start","queue_duration_s":2.360679,"scheduling_latency_s":2.360696} gitlab # [ 243.520840] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a95b31147550cb9ccc1ea61","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a95b31147550cb9ccc1ea61: done: 0.01479 sec","job_status":"done","queue_duration_s":2.360679,"scheduling_latency_s":2.360696,"redis_calls":13,"redis_duration_s":0.001585,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01479,"completed_at":"2026-09-23T19:21:32.456Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.559531] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0dd1fdba4b9e603c68f23fd","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0dd1fdba4b9e603c68f23fd: start","job_status":"start","queue_duration_s":2.376526,"scheduling_latency_s":2.376543} gitlab # [ 243.572036] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0dd1fdba4b9e603c68f23fd","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0dd1fdba4b9e603c68f23fd: done: 0.015199 sec","job_status":"done","queue_duration_s":2.376526,"scheduling_latency_s":2.376543,"redis_calls":13,"redis_duration_s":0.001412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015199,"completed_at":"2026-09-23T19:21:32.472Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.610864] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad241e6633fb3da72bcd58d0","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad241e6633fb3da72bcd58d0: start","job_status":"start","queue_duration_s":2.392827,"scheduling_latency_s":2.392843} gitlab # [ 243.623312] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad241e6633fb3da72bcd58d0","created_at":"2026-09-23T19:21:29.435Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-e684cc2244704453","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1da025a7b2485b6e279e380ca319b904","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.081Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad241e6633fb3da72bcd58d0: done: 0.015263 sec","job_status":"done","queue_duration_s":2.392827,"scheduling_latency_s":2.392843,"redis_calls":13,"redis_duration_s":0.0014329999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015263,"completed_at":"2026-09-23T19:21:32.489Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.662299] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9448d86bf1c2449f2d4e4660","created_at":"2026-09-23T19:21:30.072Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e61c0c10bc23837a25c8f9655a06013","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:21:30.098Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9448d86bf1c2449f2d4e4660: start","job_status":"start","queue_duration_s":2.392425,"scheduling_latency_s":2.392443} gitlab # [ 243.675503] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9448d86bf1c2449f2d4e4660","created_at":"2026-09-23T19:21:30.072Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e61c0c10bc23837a25c8f9655a06013","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:21:30.098Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9448d86bf1c2449f2d4e4660: done: 0.03738 sec","job_status":"done","queue_duration_s":2.392425,"scheduling_latency_s":2.392443,"redis_calls":10,"redis_duration_s":0.001329,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023944,"duration_s":0.03738,"completed_at":"2026-09-23T19:21:32.527Z","load_balancing_strategy":"primary","db_duration_s":0.008295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.717813] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9b410a78f8386e81287bee6d","created_at":"2026-09-23T19:21:30.130Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a41e4ab13a711481b96f56bb5810d5c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.151Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9b410a78f8386e81287bee6d: start","job_status":"start","queue_duration_s":2.377685,"scheduling_latency_s":2.377724} gitlab # [ 243.731655] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9b410a78f8386e81287bee6d","created_at":"2026-09-23T19:21:30.130Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a41e4ab13a711481b96f56bb5810d5c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.151Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9b410a78f8386e81287bee6d: done: 0.026155 sec","job_status":"done","queue_duration_s":2.377685,"scheduling_latency_s":2.377724,"redis_calls":10,"redis_duration_s":0.001489,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024382,"duration_s":0.026155,"completed_at":"2026-09-23T19:21:32.555Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.773530] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.556Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"43d3128547897ec265f91835","created_at":"2026-09-23T19:21:30.168Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce56b59e8eb8254b9669d4a6ae2beba8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.196Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-43d3128547897ec265f91835: start","job_status":"start","queue_duration_s":2.359804,"scheduling_latency_s":2.359823} gitlab # [ 243.787621] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"43d3128547897ec265f91835","created_at":"2026-09-23T19:21:30.168Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce56b59e8eb8254b9669d4a6ae2beba8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.196Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-43d3128547897ec265f91835: done: 0.199285 sec","job_status":"done","queue_duration_s":2.359804,"scheduling_latency_s":2.359823,"redis_calls":12,"redis_duration_s":0.0018189999999999999,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000282,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000802,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.171696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047988,"exclusive_lock_hold_duration_s":0.1782861,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.199285,"completed_at":"2026-09-23T19:21:32.755Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023598,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} gitlab # [ 243.831341] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"11fe4d1582c373e76f7a5041","created_at":"2026-09-23T19:21:30.208Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"df4eb021505aa8de17f6bdadf0f106ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.239Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-11fe4d1582c373e76f7a5041: start","job_status":"start","queue_duration_s":2.518073,"scheduling_latency_s":2.518094} gitlab # [ 243.845793] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.770Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 243.848938] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"11fe4d1582c373e76f7a5041","created_at":"2026-09-23T19:21:30.208Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"df4eb021505aa8de17f6bdadf0f106ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.239Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-11fe4d1582c373e76f7a5041: done: 0.020043 sec","job_status":"done","queue_duration_s":2.518073,"scheduling_latency_s":2.518094,"redis_calls":10,"redis_duration_s":0.001425,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000323,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023374,"duration_s":0.020043,"completed_at":"2026-09-23T19:21:32.777Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000864,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.890539] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"976dd255c431e38bf39716f9","created_at":"2026-09-23T19:21:30.289Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99b8f69b1360cf1469fe9692aefddafd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.318Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-976dd255c431e38bf39716f9: start","job_status":"start","queue_duration_s":2.459863,"scheduling_latency_s":2.45988} gitlab # [ 243.906395] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"976dd255c431e38bf39716f9","created_at":"2026-09-23T19:21:30.289Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99b8f69b1360cf1469fe9692aefddafd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.318Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-976dd255c431e38bf39716f9: done: 0.06634 sec","job_status":"done","queue_duration_s":2.459863,"scheduling_latency_s":2.45988,"redis_calls":12,"redis_duration_s":0.001823,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000716,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053806,"exclusive_lock_hold_duration_s":0.024931500000000002,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.06634,"completed_at":"2026-09-23T19:21:32.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036299,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.951099] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.846Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6918465ff09a8a5d2a2538dd","created_at":"2026-09-23T19:21:30.358Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"641e48dc483a77ce153e318aa76f4761","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.370Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6918465ff09a8a5d2a2538dd: start","job_status":"start","queue_duration_s":2.476025,"scheduling_latency_s":2.476044} gitlab # [ 243.965309] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.886Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6918465ff09a8a5d2a2538dd","created_at":"2026-09-23T19:21:30.358Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"641e48dc483a77ce153e318aa76f4761","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.370Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6918465ff09a8a5d2a2538dd: done: 0.040148 sec","job_status":"done","queue_duration_s":2.476025,"scheduling_latency_s":2.476044,"redis_calls":16,"redis_duration_s":0.002401,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001237,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027514,"duration_s":0.040148,"completed_at":"2026-09-23T19:21:32.886Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.006940] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.887Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0db5032bb1c7a7413346180","created_at":"2026-09-23T19:21:31.567Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-430966409d9d438c","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.582Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0db5032bb1c7a7413346180: start","job_status":"start","queue_duration_s":1.305726,"scheduling_latency_s":1.305746} gitlab # [ 244.019204] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.906Z","message":"No existing merge request to be cleaned up."} gitlab # [ 244.021304] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.922Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0db5032bb1c7a7413346180","created_at":"2026-09-23T19:21:31.567Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-430966409d9d438c","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.582Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0db5032bb1c7a7413346180: done: 0.034216 sec","job_status":"done","queue_duration_s":1.305726,"scheduling_latency_s":1.305746,"redis_calls":13,"redis_duration_s":0.001947,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026818,"duration_s":0.034216,"completed_at":"2026-09-23T19:21:32.921Z","load_balancing_strategy":"primary","db_duration_s":0.005614,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.061249] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.923Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a250877626598237f9cf1db4","created_at":"2026-09-23T19:21:31.567Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-430966409d9d438c","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.582Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a250877626598237f9cf1db4: start","job_status":"start","queue_duration_s":1.341088,"scheduling_latency_s":1.341106} gitlab # [ 244.073172] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.934Z","message":"No existing merge request to be cleaned up."} gitlab # [ 244.074835] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.943Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a250877626598237f9cf1db4","created_at":"2026-09-23T19:21:31.567Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-430966409d9d438c","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.582Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a250877626598237f9cf1db4: done: 0.020709 sec","job_status":"done","queue_duration_s":1.341088,"scheduling_latency_s":1.341106,"redis_calls":10,"redis_duration_s":0.0014819999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000841,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020709,"completed_at":"2026-09-23T19:21:32.943Z","load_balancing_strategy":"primary","db_duration_s":0.002529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.114356] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.945Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d535e6a492570aadb8a7463d","created_at":"2026-09-23T19:21:31.567Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-430966409d9d438c","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.582Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d535e6a492570aadb8a7463d: start","job_status":"start","queue_duration_s":1.362911,"scheduling_latency_s":1.362928} gitlab # [ 244.126740] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.956Z","message":"No existing merge request to be cleaned up."} gitlab # [ 244.128424] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.965Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d535e6a492570aadb8a7463d","created_at":"2026-09-23T19:21:31.567Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-430966409d9d438c","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.582Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d535e6a492570aadb8a7463d: done: 0.020863 sec","job_status":"done","queue_duration_s":1.362911,"scheduling_latency_s":1.362928,"redis_calls":10,"redis_duration_s":0.0014620000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000854,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020863,"completed_at":"2026-09-23T19:21:32.965Z","load_balancing_strategy":"primary","db_duration_s":0.002654,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.166897] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.966Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6054c808bebf14cb9ccb0c41","created_at":"2026-09-23T19:21:31.567Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-430966409d9d438c","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.582Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6054c808bebf14cb9ccb0c41: start","job_status":"start","queue_duration_s":1.384833,"scheduling_latency_s":1.384852} gitlab # [ 244.179276] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.978Z","message":"No existing merge request to be cleaned up."} gitlab # [ 244.181001] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:21:32.986Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6054c808bebf14cb9ccb0c41","created_at":"2026-09-23T19:21:31.567Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-430966409d9d438c","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"05095d71fe80bebe35546692b8849fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.582Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6054c808bebf14cb9ccb0c41: done: 0.019585 sec","job_status":"done","queue_duration_s":1.384833,"scheduling_latency_s":1.384852,"redis_calls":10,"redis_duration_s":0.001311,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000754,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019585,"completed_at":"2026-09-23T19:21:32.986Z","load_balancing_strategy":"primary","db_duration_s":0.002285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.065556] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:03.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"30da3b9b8aac447e4028dbe8","created_at":"2026-09-23T19:22:03.838Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:03.856Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-30da3b9b8aac447e4028dbe8: start","job_status":"start","queue_duration_s":0.003362,"scheduling_latency_s":0.003422} gitlab # [ 273.707044] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"30da3b9b8aac447e4028dbe8","created_at":"2026-09-23T19:22:03.838Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:03.856Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-30da3b9b8aac447e4028dbe8: done: 0.643101 sec","job_status":"done","queue_duration_s":0.003362,"scheduling_latency_s":0.003422,"redis_calls":61,"redis_duration_s":0.243529,"redis_read_bytes":112,"redis_write_bytes":41113,"redis_queues_calls":2,"redis_queues_duration_s":0.001649,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.235742,"redis_queues_metadata_read_bytes":108,"redis_queues_metadata_write_bytes":3328,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006138,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.267594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.643101,"completed_at":"2026-09-23T19:22:04.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.798034] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.514Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3b87b4c2b7a9fa042b422def","created_at":"2026-09-23T19:22:03.898Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"921db5866e8c8dc084480aa1a6dddfde","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:03.927Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3b87b4c2b7a9fa042b422def: start","job_status":"start","queue_duration_s":0.587243,"scheduling_latency_s":0.587297} gitlab # [ 273.869780] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3b87b4c2b7a9fa042b422def","created_at":"2026-09-23T19:22:03.898Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"921db5866e8c8dc084480aa1a6dddfde","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:03.927Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3b87b4c2b7a9fa042b422def: done: 0.150371 sec","job_status":"done","queue_duration_s":0.587243,"scheduling_latency_s":0.587297,"redis_calls":11,"redis_duration_s":0.016207,"redis_read_bytes":136,"redis_write_bytes":1288,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000343,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004776,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.011088,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.051,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05372474,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.150371,"completed_at":"2026-09-23T19:22:04.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050916,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 273.972509] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"21ae92cfca0d933e54af7fd2","created_at":"2026-09-23T19:22:03.948Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7909ce068d5161a2fd427a715f835b7b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:03.958Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-21ae92cfca0d933e54af7fd2: start","job_status":"start","queue_duration_s":0.760218,"scheduling_latency_s":0.760266} gitlab # [ 274.107957] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"21ae92cfca0d933e54af7fd2","created_at":"2026-09-23T19:22:03.948Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7909ce068d5161a2fd427a715f835b7b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:03.958Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-21ae92cfca0d933e54af7fd2: done: 0.18358 sec","job_status":"done","queue_duration_s":0.760218,"scheduling_latency_s":0.760266,"redis_calls":10,"redis_duration_s":0.010727,"redis_read_bytes":135,"redis_write_bytes":1226,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000194,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":64,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001534,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008999,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.036,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00902868,"exclusive_lock_hold_duration_s":0.12319084,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.18358,"completed_at":"2026-09-23T19:22:04.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.103596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.190482] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.906Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8d48bdd7edd37f2af2317d10","created_at":"2026-09-23T19:22:03.981Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e710c1e81ebf38ebd7b8175524d2475","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:03.992Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8d48bdd7edd37f2af2317d10: start","job_status":"start","queue_duration_s":0.913761,"scheduling_latency_s":0.913811} gitlab # [ 274.207329] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.926Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 274.211419] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8d48bdd7edd37f2af2317d10","created_at":"2026-09-23T19:22:03.981Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e710c1e81ebf38ebd7b8175524d2475","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:03.992Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8d48bdd7edd37f2af2317d10: done: 0.033127 sec","job_status":"done","queue_duration_s":0.913761,"scheduling_latency_s":0.913811,"redis_calls":7,"redis_duration_s":0.0028409999999999998,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001834,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033127,"completed_at":"2026-09-23T19:22:04.939Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.252665] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"16c0fa042491ee54526a2f88","created_at":"2026-09-23T19:22:04.018Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd70719c082bb785111886c96f66065d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.028Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-16c0fa042491ee54526a2f88: start","job_status":"start","queue_duration_s":0.913487,"scheduling_latency_s":0.913526} gitlab # [ 274.267132] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.955Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 274.271020] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.964Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"16c0fa042491ee54526a2f88","created_at":"2026-09-23T19:22:04.018Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd70719c082bb785111886c96f66065d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.028Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-16c0fa042491ee54526a2f88: done: 0.022274 sec","job_status":"done","queue_duration_s":0.913487,"scheduling_latency_s":0.913526,"redis_calls":7,"redis_duration_s":0.001598,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022274,"completed_at":"2026-09-23T19:22:04.964Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.311220] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.965Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"142126340f71bf1504e4e2a0","created_at":"2026-09-23T19:22:04.056Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2c87e5aa8b0c65b21864bf9c105f390","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:22:04.065Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-142126340f71bf1504e4e2a0: start","job_status":"start","queue_duration_s":0.900292,"scheduling_latency_s":0.900325} gitlab # [ 274.324538] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.976Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 274.327769] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.983Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"142126340f71bf1504e4e2a0","created_at":"2026-09-23T19:22:04.056Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2c87e5aa8b0c65b21864bf9c105f390","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:22:04.065Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-142126340f71bf1504e4e2a0: done: 0.017168 sec","job_status":"done","queue_duration_s":0.900292,"scheduling_latency_s":0.900325,"redis_calls":7,"redis_duration_s":0.001215,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00034,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017168,"completed_at":"2026-09-23T19:22:04.982Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.367778] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:04.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"7c5c097fad206cdc918e729d","created_at":"2026-09-23T19:22:04.086Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.114Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-7c5c097fad206cdc918e729d: start","job_status":"start","queue_duration_s":0.869606,"scheduling_latency_s":0.869636} gitlab # [ 274.382138] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"7c5c097fad206cdc918e729d","created_at":"2026-09-23T19:22:04.086Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.114Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-7c5c097fad206cdc918e729d: done: 0.078488 sec","job_status":"done","queue_duration_s":0.869606,"scheduling_latency_s":0.869636,"redis_calls":28,"redis_duration_s":0.005212,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000783,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000364,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.003062,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003863,"duration_s":0.078488,"completed_at":"2026-09-23T19:22:05.062Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.425869] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"33b51e1ce9f4b1b428fd3359","created_at":"2026-09-23T19:22:04.190Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d14d16e30559af06a38d9716b41bd9c2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.201Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-33b51e1ce9f4b1b428fd3359: start","job_status":"start","queue_duration_s":0.86275,"scheduling_latency_s":0.862783} gitlab # [ 274.440375] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"33b51e1ce9f4b1b428fd3359","created_at":"2026-09-23T19:22:04.190Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d14d16e30559af06a38d9716b41bd9c2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.201Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-33b51e1ce9f4b1b428fd3359: done: 0.017509 sec","job_status":"done","queue_duration_s":0.86275,"scheduling_latency_s":0.862783,"redis_calls":8,"redis_duration_s":0.001392,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000155,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017509,"completed_at":"2026-09-23T19:22:05.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.482129] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.083Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"9d717862484f67772cd7b604","created_at":"2026-09-23T19:22:04.238Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"412d2a84f323462d093f97aaec4b49e6","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.265Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-9d717862484f67772cd7b604: start","job_status":"start","queue_duration_s":0.818034,"scheduling_latency_s":0.818059} gitlab # [ 274.497356] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"9d717862484f67772cd7b604","created_at":"2026-09-23T19:22:04.238Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"412d2a84f323462d093f97aaec4b49e6","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.265Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-9d717862484f67772cd7b604: done: 0.033143 sec","job_status":"done","queue_duration_s":0.818034,"scheduling_latency_s":0.818059,"redis_calls":10,"redis_duration_s":0.001385,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025588,"duration_s":0.033143,"completed_at":"2026-09-23T19:22:05.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.539264] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"dae40a445223256aa27ac384","created_at":"2026-09-23T19:22:04.285Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e126506c2361e3495486f657fda8091","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.295Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-dae40a445223256aa27ac384: start","job_status":"start","queue_duration_s":0.821907,"scheduling_latency_s":0.821934} gitlab # [ 274.554655] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"dae40a445223256aa27ac384","created_at":"2026-09-23T19:22:04.285Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e126506c2361e3495486f657fda8091","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.295Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-dae40a445223256aa27ac384: done: 0.013973 sec","job_status":"done","queue_duration_s":0.821907,"scheduling_latency_s":0.821934,"redis_calls":7,"redis_duration_s":0.00098,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013973,"completed_at":"2026-09-23T19:22:05.131Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.594498] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"af9556d1ee65702f6bea3fa6","created_at":"2026-09-23T19:22:04.337Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53cbb639856ce3dd51db8f295d2262c0","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.347Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-af9556d1ee65702f6bea3fa6: start","job_status":"start","queue_duration_s":0.785723,"scheduling_latency_s":0.785971} gitlab # [ 274.651929] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.448Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"af9556d1ee65702f6bea3fa6","created_at":"2026-09-23T19:22:04.337Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53cbb639856ce3dd51db8f295d2262c0","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.347Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-af9556d1ee65702f6bea3fa6: done: 0.315422 sec","job_status":"done","queue_duration_s":0.785723,"scheduling_latency_s":0.785971,"redis_calls":562,"redis_duration_s":0.055799,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000732,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.055067,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.293703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.315422,"completed_at":"2026-09-23T19:22:05.448Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.691722] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f78ed32815bf272b1e4200fe","created_at":"2026-09-23T19:22:04.365Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8142a0d88296444bd70a1808df0d589","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.377Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f78ed32815bf272b1e4200fe: start","job_status":"start","queue_duration_s":1.072729,"scheduling_latency_s":1.072754} gitlab # [ 275.084935] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.879Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f78ed32815bf272b1e4200fe","created_at":"2026-09-23T19:22:04.365Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8142a0d88296444bd70a1808df0d589","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.377Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f78ed32815bf272b1e4200fe: done: 0.429957 sec","job_status":"done","queue_duration_s":1.072729,"scheduling_latency_s":1.072754,"redis_calls":11,"redis_duration_s":0.0024630000000000003,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000305,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001146,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.069,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.069,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.342187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050204,"exclusive_lock_hold_duration_s":0.40758612,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.429957,"completed_at":"2026-09-23T19:22:05.879Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.10355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.177908] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"af08ea6638f34be5307d5921","created_at":"2026-09-23T19:22:04.450Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d5e3021b12561d34babc5f676017523","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.458Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-af08ea6638f34be5307d5921: start","job_status":"start","queue_duration_s":1.425321,"scheduling_latency_s":1.42535} gitlab # [ 275.198892] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.931Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"af08ea6638f34be5307d5921","created_at":"2026-09-23T19:22:04.450Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d5e3021b12561d34babc5f676017523","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.458Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-af08ea6638f34be5307d5921: done: 0.047787 sec","job_status":"done","queue_duration_s":1.425321,"scheduling_latency_s":1.42535,"redis_calls":7,"redis_duration_s":0.003003,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000783,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.047787,"completed_at":"2026-09-23T19:22:05.931Z","load_balancing_strategy":"primary","db_duration_s":0.011201,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.245141] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b9df44312fc0e6e0a64fb3b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b9df44312fc0e6e0a64fb3b: start","job_status":"start","queue_duration_s":1.469203,"scheduling_latency_s":1.469248} gitlab # [ 275.257420] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b9df44312fc0e6e0a64fb3b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b9df44312fc0e6e0a64fb3b: done: 0.038087 sec","job_status":"done","queue_duration_s":1.469203,"scheduling_latency_s":1.469248,"redis_calls":13,"redis_duration_s":0.003399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002245,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038087,"completed_at":"2026-09-23T19:22:05.972Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.296123] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:05.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c7b16ef05516208e03cb7a7","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c7b16ef05516208e03cb7a7: start","job_status":"start","queue_duration_s":1.509353,"scheduling_latency_s":1.509385} gitlab # [ 275.308363] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c7b16ef05516208e03cb7a7","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c7b16ef05516208e03cb7a7: done: 0.029474 sec","job_status":"done","queue_duration_s":1.509353,"scheduling_latency_s":1.509385,"redis_calls":13,"redis_duration_s":0.00266,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001714,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029474,"completed_at":"2026-09-23T19:22:06.003Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.346725] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da9c359a4fc3008e6f6a6b7b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da9c359a4fc3008e6f6a6b7b: start","job_status":"start","queue_duration_s":1.544804,"scheduling_latency_s":1.544854} gitlab # [ 275.360129] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da9c359a4fc3008e6f6a6b7b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da9c359a4fc3008e6f6a6b7b: done: 0.056987 sec","job_status":"done","queue_duration_s":1.544804,"scheduling_latency_s":1.544854,"redis_calls":13,"redis_duration_s":0.012535000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.056987,"completed_at":"2026-09-23T19:22:06.066Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.399095] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ac8236a42bcdd07c58df04b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ac8236a42bcdd07c58df04b: start","job_status":"start","queue_duration_s":1.605643,"scheduling_latency_s":1.605692} gitlab # [ 275.411652] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ac8236a42bcdd07c58df04b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ac8236a42bcdd07c58df04b: done: 0.039467 sec","job_status":"done","queue_duration_s":1.605643,"scheduling_latency_s":1.605692,"redis_calls":13,"redis_duration_s":0.004311,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002833,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039467,"completed_at":"2026-09-23T19:22:06.110Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.450428] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc5d7aa2f2d6db32e0b6cb61","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc5d7aa2f2d6db32e0b6cb61: start","job_status":"start","queue_duration_s":1.64963,"scheduling_latency_s":1.649667} gitlab # [ 275.462817] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc5d7aa2f2d6db32e0b6cb61","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc5d7aa2f2d6db32e0b6cb61: done: 0.027783 sec","job_status":"done","queue_duration_s":1.64963,"scheduling_latency_s":1.649667,"redis_calls":13,"redis_duration_s":0.003423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001134,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002289,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027783,"completed_at":"2026-09-23T19:22:06.142Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.502078] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e79e189391b197bf32e59796","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e79e189391b197bf32e59796: start","job_status":"start","queue_duration_s":1.679227,"scheduling_latency_s":1.679256} gitlab # [ 275.514793] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e79e189391b197bf32e59796","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e79e189391b197bf32e59796: done: 0.021608 sec","job_status":"done","queue_duration_s":1.679227,"scheduling_latency_s":1.679256,"redis_calls":13,"redis_duration_s":0.002999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001016,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021608,"completed_at":"2026-09-23T19:22:06.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.553509] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d6d09666405ba441cc7999e","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d6d09666405ba441cc7999e: start","job_status":"start","queue_duration_s":1.702469,"scheduling_latency_s":1.702493} gitlab # [ 275.566901] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d6d09666405ba441cc7999e","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d6d09666405ba441cc7999e: done: 0.028315 sec","job_status":"done","queue_duration_s":1.702469,"scheduling_latency_s":1.702493,"redis_calls":13,"redis_duration_s":0.012242,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028315,"completed_at":"2026-09-23T19:22:06.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.605827] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf64dfa06c8f4b810994b006","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf64dfa06c8f4b810994b006: start","job_status":"start","queue_duration_s":1.733452,"scheduling_latency_s":1.73349} gitlab # [ 275.617982] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf64dfa06c8f4b810994b006","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf64dfa06c8f4b810994b006: done: 0.024881 sec","job_status":"done","queue_duration_s":1.733452,"scheduling_latency_s":1.73349,"redis_calls":13,"redis_duration_s":0.0035310000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024881,"completed_at":"2026-09-23T19:22:06.223Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.656932] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39cc0d423587c638c63db12e","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39cc0d423587c638c63db12e: start","job_status":"start","queue_duration_s":1.761209,"scheduling_latency_s":1.761238} gitlab # [ 275.669173] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39cc0d423587c638c63db12e","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39cc0d423587c638c63db12e: done: 0.022401 sec","job_status":"done","queue_duration_s":1.761209,"scheduling_latency_s":1.761238,"redis_calls":13,"redis_duration_s":0.004201,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022401,"completed_at":"2026-09-23T19:22:06.248Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.707639] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42bd5bc5b31786a93650096e","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42bd5bc5b31786a93650096e: start","job_status":"start","queue_duration_s":1.78534,"scheduling_latency_s":1.785366} gitlab # [ 275.720437] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42bd5bc5b31786a93650096e","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42bd5bc5b31786a93650096e: done: 0.020308 sec","job_status":"done","queue_duration_s":1.78534,"scheduling_latency_s":1.785366,"redis_calls":13,"redis_duration_s":0.002782,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001793,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020308,"completed_at":"2026-09-23T19:22:06.270Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.759256] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9311e0ef9e49256c9a029b88","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9311e0ef9e49256c9a029b88: start","job_status":"start","queue_duration_s":1.807288,"scheduling_latency_s":1.807315} gitlab # [ 275.772503] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9311e0ef9e49256c9a029b88","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9311e0ef9e49256c9a029b88: done: 0.018469 sec","job_status":"done","queue_duration_s":1.807288,"scheduling_latency_s":1.807315,"redis_calls":13,"redis_duration_s":0.002892,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018469,"completed_at":"2026-09-23T19:22:06.290Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.811642] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cea448ffc3cb835105b82f0d","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cea448ffc3cb835105b82f0d: start","job_status":"start","queue_duration_s":1.827308,"scheduling_latency_s":1.827332} gitlab # [ 275.824153] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cea448ffc3cb835105b82f0d","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cea448ffc3cb835105b82f0d: done: 0.017677 sec","job_status":"done","queue_duration_s":1.827308,"scheduling_latency_s":1.827332,"redis_calls":13,"redis_duration_s":0.002627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017677,"completed_at":"2026-09-23T19:22:06.310Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.862726] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4674b4c321f62d8f59529978","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4674b4c321f62d8f59529978: start","job_status":"start","queue_duration_s":1.846339,"scheduling_latency_s":1.846355} gitlab # [ 275.875565] setup-gitlab-start[2984]: W, [2026-09-23T19:22:06.574889 #2984] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 275.877644] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4674b4c321f62d8f59529978","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4674b4c321f62d8f59529978: done: 0.016958 sec","job_status":"done","queue_duration_s":1.846339,"scheduling_latency_s":1.846355,"redis_calls":13,"redis_duration_s":0.0028510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016958,"completed_at":"2026-09-23T19:22:06.328Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.916739] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9eee102b1bdad578ae3dacd8","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9eee102b1bdad578ae3dacd8: start","job_status":"start","queue_duration_s":1.865467,"scheduling_latency_s":1.865485} gitlab # [ 275.928787] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9eee102b1bdad578ae3dacd8","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9eee102b1bdad578ae3dacd8: done: 0.016696 sec","job_status":"done","queue_duration_s":1.865467,"scheduling_latency_s":1.865485,"redis_calls":13,"redis_duration_s":0.002698,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001765,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016696,"completed_at":"2026-09-23T19:22:06.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.967836] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2267164e17aa516cb993065f","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2267164e17aa516cb993065f: start","job_status":"start","queue_duration_s":1.883525,"scheduling_latency_s":1.883541} gitlab # [ 275.981555] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2267164e17aa516cb993065f","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2267164e17aa516cb993065f: done: 0.017983 sec","job_status":"done","queue_duration_s":1.883525,"scheduling_latency_s":1.883541,"redis_calls":13,"redis_duration_s":0.002774,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001828,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017983,"completed_at":"2026-09-23T19:22:06.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.020574] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feddc7c908af114fce854a83","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feddc7c908af114fce854a83: start","job_status":"start","queue_duration_s":1.903028,"scheduling_latency_s":1.903044} gitlab # [ 276.032545] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feddc7c908af114fce854a83","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feddc7c908af114fce854a83: done: 0.017009 sec","job_status":"done","queue_duration_s":1.903028,"scheduling_latency_s":1.903044,"redis_calls":13,"redis_duration_s":0.0029769999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000991,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017009,"completed_at":"2026-09-23T19:22:06.385Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.071648] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e673cc8748ff2418e5990b22","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e673cc8748ff2418e5990b22: start","job_status":"start","queue_duration_s":1.921394,"scheduling_latency_s":1.921409} gitlab # [ 276.084076] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e673cc8748ff2418e5990b22","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e673cc8748ff2418e5990b22: done: 0.016148 sec","job_status":"done","queue_duration_s":1.921394,"scheduling_latency_s":1.921409,"redis_calls":13,"redis_duration_s":0.003375,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002471,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016148,"completed_at":"2026-09-23T19:22:06.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.123340] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d38e2f7c6d89f14c7e101c7","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d38e2f7c6d89f14c7e101c7: start","job_status":"start","queue_duration_s":1.938888,"scheduling_latency_s":1.938903} gitlab # [ 276.136064] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d38e2f7c6d89f14c7e101c7","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d38e2f7c6d89f14c7e101c7: done: 0.016067 sec","job_status":"done","queue_duration_s":1.938888,"scheduling_latency_s":1.938903,"redis_calls":13,"redis_duration_s":0.002669,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001771,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016067,"completed_at":"2026-09-23T19:22:06.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.176296] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"348c4fda0c786b485623b2ae","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-348c4fda0c786b485623b2ae: start","job_status":"start","queue_duration_s":1.957169,"scheduling_latency_s":1.957185} gitlab # [ 276.189048] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"348c4fda0c786b485623b2ae","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-348c4fda0c786b485623b2ae: done: 0.017174 sec","job_status":"done","queue_duration_s":1.957169,"scheduling_latency_s":1.957185,"redis_calls":13,"redis_duration_s":0.0031720000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017174,"completed_at":"2026-09-23T19:22:06.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.229004] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55e48c52193b7fd9af061553","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55e48c52193b7fd9af061553: start","job_status":"start","queue_duration_s":1.977333,"scheduling_latency_s":1.977377} gitlab # [ 276.241464] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55e48c52193b7fd9af061553","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55e48c52193b7fd9af061553: done: 0.061319 sec","job_status":"done","queue_duration_s":1.977333,"scheduling_latency_s":1.977377,"redis_calls":13,"redis_duration_s":0.007952,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002817,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.061319,"completed_at":"2026-09-23T19:22:06.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.280628] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:06.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"635e39dad02c709a11998df1","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-635e39dad02c709a11998df1: start","job_status":"start","queue_duration_s":2.336565,"scheduling_latency_s":2.336599} gitlab # [ 276.422418] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"635e39dad02c709a11998df1","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-635e39dad02c709a11998df1: done: 0.416943 sec","job_status":"done","queue_duration_s":2.336565,"scheduling_latency_s":2.336599,"redis_calls":13,"redis_duration_s":0.021879,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007317,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014562,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.416943,"completed_at":"2026-09-23T19:22:07.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.485654] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4939de987d5fc439f41da877","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4939de987d5fc439f41da877: start","job_status":"start","queue_duration_s":2.762875,"scheduling_latency_s":2.762931} gitlab # [ 276.532431] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4939de987d5fc439f41da877","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4939de987d5fc439f41da877: done: 0.100416 sec","job_status":"done","queue_duration_s":2.762875,"scheduling_latency_s":2.762931,"redis_calls":13,"redis_duration_s":0.026893,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.100416,"completed_at":"2026-09-23T19:22:07.328Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.598179] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5c947b4d2fc92ee9a9fadf2","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5c947b4d2fc92ee9a9fadf2: start","job_status":"start","queue_duration_s":2.879177,"scheduling_latency_s":2.879236} gitlab # [ 276.636828] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5c947b4d2fc92ee9a9fadf2","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5c947b4d2fc92ee9a9fadf2: done: 0.08835 sec","job_status":"done","queue_duration_s":2.879177,"scheduling_latency_s":2.879236,"redis_calls":13,"redis_duration_s":0.027978,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005069,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.022909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.08835,"completed_at":"2026-09-23T19:22:07.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.691832] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d6302bb03577d1ad5cd4d52","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d6302bb03577d1ad5cd4d52: start","job_status":"start","queue_duration_s":2.973664,"scheduling_latency_s":2.973708} gitlab # [ 276.709889] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d6302bb03577d1ad5cd4d52","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d6302bb03577d1ad5cd4d52: done: 0.067335 sec","job_status":"done","queue_duration_s":2.973664,"scheduling_latency_s":2.973708,"redis_calls":13,"redis_duration_s":0.003619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002333,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.067335,"completed_at":"2026-09-23T19:22:07.505Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.753357] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c63a65b788587eec2073d74e","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c63a65b788587eec2073d74e: start","job_status":"start","queue_duration_s":3.044652,"scheduling_latency_s":3.044687} gitlab # [ 276.766250] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c63a65b788587eec2073d74e","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c63a65b788587eec2073d74e: done: 0.05248 sec","job_status":"done","queue_duration_s":3.044652,"scheduling_latency_s":3.044687,"redis_calls":13,"redis_duration_s":0.003427,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.05248,"completed_at":"2026-09-23T19:22:07.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.806000] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67b99b8a425c648d9dc7b502","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67b99b8a425c648d9dc7b502: start","job_status":"start","queue_duration_s":3.102839,"scheduling_latency_s":3.102877} gitlab # [ 276.838908] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67b99b8a425c648d9dc7b502","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67b99b8a425c648d9dc7b502: done: 0.067267 sec","job_status":"done","queue_duration_s":3.102839,"scheduling_latency_s":3.102877,"redis_calls":13,"redis_duration_s":0.003617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001226,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002391,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.067267,"completed_at":"2026-09-23T19:22:07.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.878446] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a9321f663e08dbd8e1e467a","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a9321f663e08dbd8e1e467a: start","job_status":"start","queue_duration_s":3.174187,"scheduling_latency_s":3.174232} gitlab # [ 276.905186] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a9321f663e08dbd8e1e467a","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a9321f663e08dbd8e1e467a: done: 0.060824 sec","job_status":"done","queue_duration_s":3.174187,"scheduling_latency_s":3.174232,"redis_calls":13,"redis_duration_s":0.00422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.060824,"completed_at":"2026-09-23T19:22:07.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.945497] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d507cace18fd633a98cb93b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d507cace18fd633a98cb93b: start","job_status":"start","queue_duration_s":3.240472,"scheduling_latency_s":3.240517} gitlab # [ 276.976502] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d507cace18fd633a98cb93b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d507cace18fd633a98cb93b: done: 0.066756 sec","job_status":"done","queue_duration_s":3.240472,"scheduling_latency_s":3.240517,"redis_calls":13,"redis_duration_s":0.003467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.066756,"completed_at":"2026-09-23T19:22:07.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.020576] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6248ea493ed1740ae5e91125","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6248ea493ed1740ae5e91125: start","job_status":"start","queue_duration_s":3.312688,"scheduling_latency_s":3.312725} gitlab # [ 277.103116] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6248ea493ed1740ae5e91125","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6248ea493ed1740ae5e91125: done: 0.121769 sec","job_status":"done","queue_duration_s":3.312688,"scheduling_latency_s":3.312725,"redis_calls":13,"redis_duration_s":0.003661,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001263,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002398,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.121769,"completed_at":"2026-09-23T19:22:07.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.162497] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:07.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0e2a524d2f960f663463f95","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0e2a524d2f960f663463f95: start","job_status":"start","queue_duration_s":3.441084,"scheduling_latency_s":3.441132} gitlab # [ 277.245255] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0e2a524d2f960f663463f95","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0e2a524d2f960f663463f95: done: 0.134096 sec","job_status":"done","queue_duration_s":3.441084,"scheduling_latency_s":3.441132,"redis_calls":13,"redis_duration_s":0.057425000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.05251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.134096,"completed_at":"2026-09-23T19:22:08.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.309572] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b21f4725cc060fab0d90895","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b21f4725cc060fab0d90895: start","job_status":"start","queue_duration_s":3.586855,"scheduling_latency_s":3.586905} gitlab # [ 277.345425] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b21f4725cc060fab0d90895","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b21f4725cc060fab0d90895: done: 0.088961 sec","job_status":"done","queue_duration_s":3.586855,"scheduling_latency_s":3.586905,"redis_calls":13,"redis_duration_s":0.023814,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007323,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.016491,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.088961,"completed_at":"2026-09-23T19:22:08.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.385919] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1bf92db8453729c21141142","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1bf92db8453729c21141142: start","job_status":"start","queue_duration_s":3.68646,"scheduling_latency_s":3.6865} gitlab # [ 277.445442] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1bf92db8453729c21141142","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1bf92db8453729c21141142: done: 0.088857 sec","job_status":"done","queue_duration_s":3.68646,"scheduling_latency_s":3.6865,"redis_calls":13,"redis_duration_s":0.01678,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.011701,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.088857,"completed_at":"2026-09-23T19:22:08.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.484570] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8ca0800e8d3d47f4e5219f4","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8ca0800e8d3d47f4e5219f4: start","job_status":"start","queue_duration_s":3.77972,"scheduling_latency_s":3.779768} gitlab # [ 277.526729] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8ca0800e8d3d47f4e5219f4","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8ca0800e8d3d47f4e5219f4: done: 0.077217 sec","job_status":"done","queue_duration_s":3.77972,"scheduling_latency_s":3.779768,"redis_calls":13,"redis_duration_s":0.022243,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008385,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.077217,"completed_at":"2026-09-23T19:22:08.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.565522] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27a82de5983dff49246d9541","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27a82de5983dff49246d9541: start","job_status":"start","queue_duration_s":3.868338,"scheduling_latency_s":3.868382} gitlab # [ 277.624219] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27a82de5983dff49246d9541","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27a82de5983dff49246d9541: done: 0.08593 sec","job_status":"done","queue_duration_s":3.868338,"scheduling_latency_s":3.868382,"redis_calls":13,"redis_duration_s":0.024102,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01653,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.08593,"completed_at":"2026-09-23T19:22:08.419Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.692353] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32538ef5d21248fb14a40423","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32538ef5d21248fb14a40423: start","job_status":"start","queue_duration_s":3.966735,"scheduling_latency_s":3.966783} gitlab # [ 277.706399] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32538ef5d21248fb14a40423","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32538ef5d21248fb14a40423: done: 0.051265 sec","job_status":"done","queue_duration_s":3.966735,"scheduling_latency_s":3.966783,"redis_calls":13,"redis_duration_s":0.002977,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.051265,"completed_at":"2026-09-23T19:22:08.482Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.745211] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adc0ee87fbb541e1e1facaba","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adc0ee87fbb541e1e1facaba: start","job_status":"start","queue_duration_s":4.023543,"scheduling_latency_s":4.023568} gitlab # [ 277.784556] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adc0ee87fbb541e1e1facaba","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adc0ee87fbb541e1e1facaba: done: 0.091801 sec","job_status":"done","queue_duration_s":4.023543,"scheduling_latency_s":4.023568,"redis_calls":13,"redis_duration_s":0.0034260000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001198,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.091801,"completed_at":"2026-09-23T19:22:08.580Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.867257] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e50f04e55fe740b1a6b3e921","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e50f04e55fe740b1a6b3e921: start","job_status":"start","queue_duration_s":4.198387,"scheduling_latency_s":4.198443} gitlab # [ 278.038458] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e50f04e55fe740b1a6b3e921","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e50f04e55fe740b1a6b3e921: done: 0.167757 sec","job_status":"done","queue_duration_s":4.198387,"scheduling_latency_s":4.198443,"redis_calls":13,"redis_duration_s":0.022503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.016533,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.167757,"completed_at":"2026-09-23T19:22:08.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.129489] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0e019a0efd30db9a7144bb6","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0e019a0efd30db9a7144bb6: start","job_status":"start","queue_duration_s":4.45983,"scheduling_latency_s":4.459894} gitlab # [ 278.176093] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0e019a0efd30db9a7144bb6","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0e019a0efd30db9a7144bb6: done: 0.047399 sec","job_status":"done","queue_duration_s":4.45983,"scheduling_latency_s":4.459894,"redis_calls":13,"redis_duration_s":0.003725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00136,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002365,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.047399,"completed_at":"2026-09-23T19:22:08.972Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.236036] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:08.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd586506a2ee0cd7e5a0acf3","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd586506a2ee0cd7e5a0acf3: start","job_status":"start","queue_duration_s":4.510227,"scheduling_latency_s":4.510269} gitlab # [ 278.252289] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd586506a2ee0cd7e5a0acf3","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd586506a2ee0cd7e5a0acf3: done: 0.028699 sec","job_status":"done","queue_duration_s":4.510227,"scheduling_latency_s":4.510269,"redis_calls":13,"redis_duration_s":0.0036820000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002448,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028699,"completed_at":"2026-09-23T19:22:09.003Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.293166] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2322bdb25134819b3fde77d5","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2322bdb25134819b3fde77d5: start","job_status":"start","queue_duration_s":4.541196,"scheduling_latency_s":4.541234} gitlab # [ 278.305581] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2322bdb25134819b3fde77d5","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2322bdb25134819b3fde77d5: done: 0.024558 sec","job_status":"done","queue_duration_s":4.541196,"scheduling_latency_s":4.541234,"redis_calls":13,"redis_duration_s":0.0032949999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024558,"completed_at":"2026-09-23T19:22:09.030Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.344576] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28ecf6c84c602808534677c9","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28ecf6c84c602808534677c9: start","job_status":"start","queue_duration_s":4.567708,"scheduling_latency_s":4.567743} gitlab # [ 278.356764] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28ecf6c84c602808534677c9","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28ecf6c84c602808534677c9: done: 0.023417 sec","job_status":"done","queue_duration_s":4.567708,"scheduling_latency_s":4.567743,"redis_calls":13,"redis_duration_s":0.003966,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001777,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023417,"completed_at":"2026-09-23T19:22:09.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.395294] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ca961d32e849bbf48e444ed","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ca961d32e849bbf48e444ed: start","job_status":"start","queue_duration_s":4.593289,"scheduling_latency_s":4.593324} gitlab # [ 278.407775] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ca961d32e849bbf48e444ed","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ca961d32e849bbf48e444ed: done: 0.020903 sec","job_status":"done","queue_duration_s":4.593289,"scheduling_latency_s":4.593324,"redis_calls":13,"redis_duration_s":0.0030039999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020903,"completed_at":"2026-09-23T19:22:09.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.448207] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d5c964fa9c8ee54623f8d5b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d5c964fa9c8ee54623f8d5b: start","job_status":"start","queue_duration_s":4.615968,"scheduling_latency_s":4.616014} gitlab # [ 278.460205] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d5c964fa9c8ee54623f8d5b","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d5c964fa9c8ee54623f8d5b: done: 0.018796 sec","job_status":"done","queue_duration_s":4.615968,"scheduling_latency_s":4.616014,"redis_calls":13,"redis_duration_s":0.0029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018796,"completed_at":"2026-09-23T19:22:09.099Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.498932] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2585bed1ebe77bb9859b4fb","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2585bed1ebe77bb9859b4fb: start","job_status":"start","queue_duration_s":4.636331,"scheduling_latency_s":4.63636} gitlab # [ 278.511304] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2585bed1ebe77bb9859b4fb","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2585bed1ebe77bb9859b4fb: done: 0.017568 sec","job_status":"done","queue_duration_s":4.636331,"scheduling_latency_s":4.63636,"redis_calls":13,"redis_duration_s":0.0029679999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017568,"completed_at":"2026-09-23T19:22:09.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.550357] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7431fefe657b22442beafb04","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7431fefe657b22442beafb04: start","job_status":"start","queue_duration_s":4.655557,"scheduling_latency_s":4.655586} gitlab # [ 278.562823] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7431fefe657b22442beafb04","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7431fefe657b22442beafb04: done: 0.017419 sec","job_status":"done","queue_duration_s":4.655557,"scheduling_latency_s":4.655586,"redis_calls":13,"redis_duration_s":0.003013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017419,"completed_at":"2026-09-23T19:22:09.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.601720] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d8a77dea9646e066bbcb6ac","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d8a77dea9646e066bbcb6ac: start","job_status":"start","queue_duration_s":4.674511,"scheduling_latency_s":4.674531} gitlab # [ 278.614110] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d8a77dea9646e066bbcb6ac","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d8a77dea9646e066bbcb6ac: done: 0.016576 sec","job_status":"done","queue_duration_s":4.674511,"scheduling_latency_s":4.674531,"redis_calls":13,"redis_duration_s":0.003114,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001344,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016576,"completed_at":"2026-09-23T19:22:09.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.654268] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b5fdf8e2623c5a322302606","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b5fdf8e2623c5a322302606: start","job_status":"start","queue_duration_s":4.692594,"scheduling_latency_s":4.692615} gitlab # [ 278.666677] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b5fdf8e2623c5a322302606","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b5fdf8e2623c5a322302606: done: 0.016351 sec","job_status":"done","queue_duration_s":4.692594,"scheduling_latency_s":4.692615,"redis_calls":13,"redis_duration_s":0.002797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016351,"completed_at":"2026-09-23T19:22:09.174Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.705264] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d095246054ddb31cf6b003f","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d095246054ddb31cf6b003f: start","job_status":"start","queue_duration_s":4.710553,"scheduling_latency_s":4.710574} gitlab # [ 278.717500] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d095246054ddb31cf6b003f","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d095246054ddb31cf6b003f: done: 0.015952 sec","job_status":"done","queue_duration_s":4.710553,"scheduling_latency_s":4.710574,"redis_calls":13,"redis_duration_s":0.002695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001751,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015952,"completed_at":"2026-09-23T19:22:09.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.756340] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18bf3152716dbf1c6de78d4a","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18bf3152716dbf1c6de78d4a: start","job_status":"start","queue_duration_s":4.728047,"scheduling_latency_s":4.728066} gitlab # [ 278.768571] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18bf3152716dbf1c6de78d4a","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18bf3152716dbf1c6de78d4a: done: 0.016145 sec","job_status":"done","queue_duration_s":4.728047,"scheduling_latency_s":4.728066,"redis_calls":13,"redis_duration_s":0.002738,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016145,"completed_at":"2026-09-23T19:22:09.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.807078] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3441d3882a5afd74c32996ff","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3441d3882a5afd74c32996ff: start","job_status":"start","queue_duration_s":4.74575,"scheduling_latency_s":4.745769} gitlab # [ 278.819453] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3441d3882a5afd74c32996ff","created_at":"2026-09-23T19:22:03.890Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-9d50c16e843643db","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d61378da8291ca11e0374dd58a800de9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.465Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3441d3882a5afd74c32996ff: done: 0.015451 sec","job_status":"done","queue_duration_s":4.74575,"scheduling_latency_s":4.745769,"redis_calls":13,"redis_duration_s":0.002742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015451,"completed_at":"2026-09-23T19:22:09.226Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.859621] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f2e8db6650b5f374a1af3cf6","created_at":"2026-09-23T19:22:04.484Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d3d9adf5ee51dabb8337e50cd1f65a3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.496Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f2e8db6650b5f374a1af3cf6: start","job_status":"start","queue_duration_s":4.731351,"scheduling_latency_s":4.731367} gitlab # [ 278.873730] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.246Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f2e8db6650b5f374a1af3cf6","created_at":"2026-09-23T19:22:04.484Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d3d9adf5ee51dabb8337e50cd1f65a3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.496Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f2e8db6650b5f374a1af3cf6: done: 0.018523 sec","job_status":"done","queue_duration_s":4.731351,"scheduling_latency_s":4.731367,"redis_calls":9,"redis_duration_s":0.002293,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001199,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055178,"exclusive_lock_hold_duration_s":0.00441964,"duration_s":0.018523,"completed_at":"2026-09-23T19:22:09.246Z","load_balancing_strategy":"primary","db_duration_s":0.001447,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.913820] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.247Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"87c53bcb79b6ca4414279be5","created_at":"2026-09-23T19:22:04.516Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bca8e1d5e33857a4a73eebfff68adb18","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.524Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-87c53bcb79b6ca4414279be5: start","job_status":"start","queue_duration_s":4.723606,"scheduling_latency_s":4.723621} gitlab # [ 278.928392] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"87c53bcb79b6ca4414279be5","created_at":"2026-09-23T19:22:04.516Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bca8e1d5e33857a4a73eebfff68adb18","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.524Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-87c53bcb79b6ca4414279be5: done: 0.01849 sec","job_status":"done","queue_duration_s":4.723606,"scheduling_latency_s":4.723621,"redis_calls":9,"redis_duration_s":0.002736,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001523,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005379,"exclusive_lock_hold_duration_s":0.0035076,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.01849,"completed_at":"2026-09-23T19:22:09.266Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.969585] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.267Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a08f62d1b92896a0526d41cb","created_at":"2026-09-23T19:22:04.584Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.590Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a08f62d1b92896a0526d41cb: start","job_status":"start","queue_duration_s":4.677897,"scheduling_latency_s":4.677915} gitlab # [ 278.982936] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a08f62d1b92896a0526d41cb","created_at":"2026-09-23T19:22:04.584Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.590Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a08f62d1b92896a0526d41cb: done: 0.026804 sec","job_status":"done","queue_duration_s":4.677897,"scheduling_latency_s":4.677915,"redis_calls":14,"redis_duration_s":0.003222,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000358,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00215,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000714,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026804,"completed_at":"2026-09-23T19:22:09.294Z","load_balancing_strategy":"primary","db_duration_s":0.001318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.024141] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.296Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8d4bcab3e59cc78d75271087","created_at":"2026-09-23T19:22:04.613Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef49ab89cc6a89df37b6eb7c1a8c0295","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:22:04.618Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8d4bcab3e59cc78d75271087: start","job_status":"start","queue_duration_s":4.6774,"scheduling_latency_s":4.677416} gitlab # [ 279.038653] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.303Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 279.042495] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8d4bcab3e59cc78d75271087","created_at":"2026-09-23T19:22:04.613Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef49ab89cc6a89df37b6eb7c1a8c0295","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:22:04.618Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8d4bcab3e59cc78d75271087: done: 0.207455 sec","job_status":"done","queue_duration_s":4.6774,"scheduling_latency_s":4.677416,"redis_calls":7,"redis_duration_s":0.006815,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004693,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.207455,"completed_at":"2026-09-23T19:22:09.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.081532] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3befdd5da7504dcc156a69fe","created_at":"2026-09-23T19:22:04.647Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e872f71192c5e7f58b64b84b2a01879","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.659Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3befdd5da7504dcc156a69fe: start","job_status":"start","queue_duration_s":4.847561,"scheduling_latency_s":4.847611} gitlab # [ 279.096148] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.526Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 279.099469] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3befdd5da7504dcc156a69fe","created_at":"2026-09-23T19:22:04.647Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e872f71192c5e7f58b64b84b2a01879","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.659Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3befdd5da7504dcc156a69fe: done: 0.032882 sec","job_status":"done","queue_duration_s":4.847561,"scheduling_latency_s":4.847611,"redis_calls":7,"redis_duration_s":0.00252,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000649,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032882,"completed_at":"2026-09-23T19:22:09.539Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.143471] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.543Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9cfa636fcb4e3e4bfc9af01d","created_at":"2026-09-23T19:22:04.718Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e77cdc945059ff61552b96785908ccf2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:22:04.740Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9cfa636fcb4e3e4bfc9af01d: start","job_status":"start","queue_duration_s":4.802493,"scheduling_latency_s":4.802538} gitlab # [ 279.156755] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9cfa636fcb4e3e4bfc9af01d","created_at":"2026-09-23T19:22:04.718Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e77cdc945059ff61552b96785908ccf2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:22:04.740Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9cfa636fcb4e3e4bfc9af01d: done: 0.047219 sec","job_status":"done","queue_duration_s":4.802493,"scheduling_latency_s":4.802538,"redis_calls":7,"redis_duration_s":0.002805,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001822,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047219,"completed_at":"2026-09-23T19:22:09.590Z","load_balancing_strategy":"primary","db_duration_s":0.002917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.195890] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8137f0b5776db57fffeb7d98","created_at":"2026-09-23T19:22:04.750Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1dd4955bd0a65eb19ac6dd3a8c27f0d2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.754Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8137f0b5776db57fffeb7d98: start","job_status":"start","queue_duration_s":4.83804,"scheduling_latency_s":4.838068} gitlab # [ 279.210046] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8137f0b5776db57fffeb7d98","created_at":"2026-09-23T19:22:04.750Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1dd4955bd0a65eb19ac6dd3a8c27f0d2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.754Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8137f0b5776db57fffeb7d98: done: 0.022906 sec","job_status":"done","queue_duration_s":4.83804,"scheduling_latency_s":4.838068,"redis_calls":7,"redis_duration_s":0.001959,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022906,"completed_at":"2026-09-23T19:22:09.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.250904] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"54bcbb993f0dbdb27aa35ad5","created_at":"2026-09-23T19:22:04.763Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5442c904bb0eb62ae617e3eeaa188dd2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.770Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-54bcbb993f0dbdb27aa35ad5: start","job_status":"start","queue_duration_s":4.846455,"scheduling_latency_s":4.84648} gitlab # [ 279.265614] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.627Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 279.268774] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"54bcbb993f0dbdb27aa35ad5","created_at":"2026-09-23T19:22:04.763Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5442c904bb0eb62ae617e3eeaa188dd2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.770Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-54bcbb993f0dbdb27aa35ad5: done: 0.016366 sec","job_status":"done","queue_duration_s":4.846455,"scheduling_latency_s":4.84648,"redis_calls":7,"redis_duration_s":0.001886,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001351,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016366,"completed_at":"2026-09-23T19:22:09.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.308764] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"0acc31ef44bc7899219c93ce","created_at":"2026-09-23T19:22:04.780Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"caaf00c0b183a7b5afb4faa55b2f093b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.810Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-0acc31ef44bc7899219c93ce: start","job_status":"start","queue_duration_s":4.824172,"scheduling_latency_s":4.824192} gitlab # [ 279.323094] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"0acc31ef44bc7899219c93ce","created_at":"2026-09-23T19:22:04.780Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"caaf00c0b183a7b5afb4faa55b2f093b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.810Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-0acc31ef44bc7899219c93ce: done: 0.027242 sec","job_status":"done","queue_duration_s":4.824172,"scheduling_latency_s":4.824192,"redis_calls":12,"redis_duration_s":0.003543,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000516,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001845,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001182,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063018,"exclusive_lock_hold_duration_s":0.00559682,"duration_s":0.027242,"completed_at":"2026-09-23T19:22:09.662Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002452,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.365502] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.663Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"dd264d25d0bfb945130d5b93","created_at":"2026-09-23T19:22:04.816Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e6a158e34077cb635b97b9dd121fbb2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.825Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-dd264d25d0bfb945130d5b93: start","job_status":"start","queue_duration_s":4.837871,"scheduling_latency_s":4.837887} gitlab # [ 279.378489] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.683Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"dd264d25d0bfb945130d5b93","created_at":"2026-09-23T19:22:04.816Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e6a158e34077cb635b97b9dd121fbb2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.825Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-dd264d25d0bfb945130d5b93: done: 0.019412 sec","job_status":"done","queue_duration_s":4.837871,"scheduling_latency_s":4.837887,"redis_calls":12,"redis_duration_s":0.002893,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000512,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001346,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033034,"duration_s":0.019412,"completed_at":"2026-09-23T19:22:09.683Z","load_balancing_strategy":"primary","db_duration_s":0.000316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.420285] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.684Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e7cbdd9394a934b7b9188843","created_at":"2026-09-23T19:22:04.828Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7ed775c67c42fe3813d05f1490e08a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.831Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e7cbdd9394a934b7b9188843: start","job_status":"start","queue_duration_s":4.852911,"scheduling_latency_s":4.852928} gitlab # [ 279.434643] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e7cbdd9394a934b7b9188843","created_at":"2026-09-23T19:22:04.828Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7ed775c67c42fe3813d05f1490e08a5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.831Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e7cbdd9394a934b7b9188843: done: 0.035301 sec","job_status":"done","queue_duration_s":4.852911,"scheduling_latency_s":4.852928,"redis_calls":9,"redis_duration_s":0.0025139999999999997,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00074296,"exclusive_lock_hold_duration_s":0.0124634,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.035301,"completed_at":"2026-09-23T19:22:09.719Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.478884] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.721Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c6ddf02999bf6414feb10c0b","created_at":"2026-09-23T19:22:04.843Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c99b20ae863ad1978fdabb573b4960b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.846Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c6ddf02999bf6414feb10c0b: start","job_status":"start","queue_duration_s":4.875117,"scheduling_latency_s":4.875134} gitlab # [ 279.492989] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.741Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c6ddf02999bf6414feb10c0b","created_at":"2026-09-23T19:22:04.843Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-6bdd1106f2cd49ff","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c99b20ae863ad1978fdabb573b4960b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:04.846Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c6ddf02999bf6414feb10c0b: done: 0.019572 sec","job_status":"done","queue_duration_s":4.875117,"scheduling_latency_s":4.875134,"redis_calls":7,"redis_duration_s":0.001524,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019572,"completed_at":"2026-09-23T19:22:09.741Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.532643] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3629e81d3e2ddc1c1a2f410a","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3629e81d3e2ddc1c1a2f410a: start","job_status":"start","queue_duration_s":4.689935,"scheduling_latency_s":4.689952} gitlab # [ 279.545648] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3629e81d3e2ddc1c1a2f410a","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3629e81d3e2ddc1c1a2f410a: done: 0.06848 sec","job_status":"done","queue_duration_s":4.689935,"scheduling_latency_s":4.689952,"redis_calls":16,"redis_duration_s":0.005624,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003002,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00059418,"duration_s":0.06848,"completed_at":"2026-09-23T19:22:09.811Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.586106] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4895f7c3d1504a8328d98e39","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4895f7c3d1504a8328d98e39: start","job_status":"start","queue_duration_s":4.762057,"scheduling_latency_s":4.762102} gitlab # [ 279.599277] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4895f7c3d1504a8328d98e39","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4895f7c3d1504a8328d98e39: done: 0.03876 sec","job_status":"done","queue_duration_s":4.762057,"scheduling_latency_s":4.762102,"redis_calls":13,"redis_duration_s":0.004148,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001277,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03876,"completed_at":"2026-09-23T19:22:09.853Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001744,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.638163] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"fb784e53bded2ef160419d3e","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-fb784e53bded2ef160419d3e: start","job_status":"start","queue_duration_s":4.803615,"scheduling_latency_s":4.80365} gitlab # [ 279.652543] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"fb784e53bded2ef160419d3e","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-fb784e53bded2ef160419d3e: done: 0.031163 sec","job_status":"done","queue_duration_s":4.803615,"scheduling_latency_s":4.80365,"redis_calls":13,"redis_duration_s":0.003543,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002379,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031163,"completed_at":"2026-09-23T19:22:09.887Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.691870] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"93d22c8e125d25ab8400ddd2","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-93d22c8e125d25ab8400ddd2: start","job_status":"start","queue_duration_s":4.837368,"scheduling_latency_s":4.8374} gitlab # [ 279.705417] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"93d22c8e125d25ab8400ddd2","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-93d22c8e125d25ab8400ddd2: done: 0.028325 sec","job_status":"done","queue_duration_s":4.837368,"scheduling_latency_s":4.8374,"redis_calls":13,"redis_duration_s":0.003398,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001125,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028325,"completed_at":"2026-09-23T19:22:09.918Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.744291] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"72c15bbaab31afccaf0b456a","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-72c15bbaab31afccaf0b456a: start","job_status":"start","queue_duration_s":4.867602,"scheduling_latency_s":4.867628} gitlab # [ 279.757250] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"72c15bbaab31afccaf0b456a","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-72c15bbaab31afccaf0b456a: done: 0.030535 sec","job_status":"done","queue_duration_s":4.867602,"scheduling_latency_s":4.867628,"redis_calls":13,"redis_duration_s":0.0023829999999999997,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001527,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030535,"completed_at":"2026-09-23T19:22:09.951Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.796574] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8ed58da1a62accea8c211f3d","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8ed58da1a62accea8c211f3d: start","job_status":"start","queue_duration_s":4.89984,"scheduling_latency_s":4.899866} gitlab # [ 279.809303] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8ed58da1a62accea8c211f3d","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8ed58da1a62accea8c211f3d: done: 0.024377 sec","job_status":"done","queue_duration_s":4.89984,"scheduling_latency_s":4.899866,"redis_calls":13,"redis_duration_s":0.002161,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000728,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001433,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024377,"completed_at":"2026-09-23T19:22:09.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.849730] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7c75c4366fb6358ceb9a7cc3","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7c75c4366fb6358ceb9a7cc3: start","job_status":"start","queue_duration_s":4.925734,"scheduling_latency_s":4.925757} gitlab # [ 279.862653] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7c75c4366fb6358ceb9a7cc3","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7c75c4366fb6358ceb9a7cc3: done: 0.01905 sec","job_status":"done","queue_duration_s":4.925734,"scheduling_latency_s":4.925757,"redis_calls":13,"redis_duration_s":0.0017959999999999999,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01905,"completed_at":"2026-09-23T19:22:09.997Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.901703] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:09.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"51a33645ace49f0cb9cef92b","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-51a33645ace49f0cb9cef92b: start","job_status":"start","queue_duration_s":4.946067,"scheduling_latency_s":4.946088} gitlab # [ 279.914623] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"51a33645ace49f0cb9cef92b","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-51a33645ace49f0cb9cef92b: done: 0.018022 sec","job_status":"done","queue_duration_s":4.946067,"scheduling_latency_s":4.946088,"redis_calls":13,"redis_duration_s":0.001761,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001154,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018022,"completed_at":"2026-09-23T19:22:10.017Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.953996] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b2cba8922c23bb10f647867c","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b2cba8922c23bb10f647867c: start","job_status":"start","queue_duration_s":4.965381,"scheduling_latency_s":4.9654} gitlab # [ 279.967207] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b2cba8922c23bb10f647867c","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b2cba8922c23bb10f647867c: done: 0.017425 sec","job_status":"done","queue_duration_s":4.965381,"scheduling_latency_s":4.9654,"redis_calls":13,"redis_duration_s":0.0017929999999999999,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017425,"completed_at":"2026-09-23T19:22:10.035Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.006438] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b26733e0e772294ab55d2fcc","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b26733e0e772294ab55d2fcc: start","job_status":"start","queue_duration_s":4.983992,"scheduling_latency_s":4.98401} gitlab # [ 280.019576] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b26733e0e772294ab55d2fcc","created_at":"2026-09-23T19:22:05.021Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-612722d75c6d4c06","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"236cd6ede65df233a15598dcd7196d50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:05.053Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b26733e0e772294ab55d2fcc: done: 0.016393 sec","job_status":"done","queue_duration_s":4.983992,"scheduling_latency_s":4.98401,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016393,"completed_at":"2026-09-23T19:22:10.053Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.059965] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.054Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"297266df13605a1e5006fd90","created_at":"2026-09-23T19:22:09.279Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-4c591ccee3e647d7","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:09.289Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-297266df13605a1e5006fd90: start","job_status":"start","queue_duration_s":0.765325,"scheduling_latency_s":0.765342} gitlab # [ 280.072407] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.066Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.074029] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.075Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"297266df13605a1e5006fd90","created_at":"2026-09-23T19:22:09.279Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-4c591ccee3e647d7","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:09.289Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-297266df13605a1e5006fd90: done: 0.021162 sec","job_status":"done","queue_duration_s":0.765325,"scheduling_latency_s":0.765342,"redis_calls":10,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000803,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021162,"completed_at":"2026-09-23T19:22:10.075Z","load_balancing_strategy":"primary","db_duration_s":0.002875,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.112179] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.076Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"89612abc89164b70321f83b9","created_at":"2026-09-23T19:22:09.279Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-4c591ccee3e647d7","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:09.289Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-89612abc89164b70321f83b9: start","job_status":"start","queue_duration_s":0.787602,"scheduling_latency_s":0.78762} gitlab # [ 280.124368] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.087Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.126015] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.096Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"89612abc89164b70321f83b9","created_at":"2026-09-23T19:22:09.279Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-4c591ccee3e647d7","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:09.289Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-89612abc89164b70321f83b9: done: 0.019274 sec","job_status":"done","queue_duration_s":0.787602,"scheduling_latency_s":0.78762,"redis_calls":10,"redis_duration_s":0.001352,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000793,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019274,"completed_at":"2026-09-23T19:22:10.096Z","load_balancing_strategy":"primary","db_duration_s":0.002073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.164202] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.097Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"88fc301af1656050b550a501","created_at":"2026-09-23T19:22:09.279Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-4c591ccee3e647d7","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:09.289Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-88fc301af1656050b550a501: start","job_status":"start","queue_duration_s":0.807982,"scheduling_latency_s":0.807999} gitlab # [ 280.176175] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.107Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.177852] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.116Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"88fc301af1656050b550a501","created_at":"2026-09-23T19:22:09.279Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-4c591ccee3e647d7","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:09.289Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-88fc301af1656050b550a501: done: 0.018924 sec","job_status":"done","queue_duration_s":0.807982,"scheduling_latency_s":0.807999,"redis_calls":10,"redis_duration_s":0.001337,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018924,"completed_at":"2026-09-23T19:22:10.116Z","load_balancing_strategy":"primary","db_duration_s":0.001904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.216261] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.117Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"af698a6625586810126593a5","created_at":"2026-09-23T19:22:09.279Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-4c591ccee3e647d7","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:09.289Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-af698a6625586810126593a5: start","job_status":"start","queue_duration_s":0.828103,"scheduling_latency_s":0.828122} gitlab # [ 280.228498] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.127Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.230147] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:10.136Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"af698a6625586810126593a5","created_at":"2026-09-23T19:22:09.279Z","trace_propagation_headers":{"sentry-trace":"e89a0ead963f427e830c6b1652054f8c-4c591ccee3e647d7","baggage":"sentry-trace_id=e89a0ead963f427e830c6b1652054f8c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8536b33c9d03c473e0d98521ab047a31","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:22:09.289Z","job_size_bytes":2,"pid":2024,"sidekiq_tid":"mao0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-af698a6625586810126593a5: done: 0.018867 sec","job_status":"done","queue_duration_s":0.828103,"scheduling_latency_s":0.828122,"redis_calls":10,"redis_duration_s":0.0013089999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000771,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018867,"completed_at":"2026-09-23T19:22:10.136Z","load_balancing_strategy":"primary","db_duration_s":0.001913,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.267313] redis[793]: 10 changes in 300 seconds. Saving... gitlab # [ 309.271085] redis[793]: Background saving started by pid 3308 gitlab # [ 309.278237] redis[3308]: BGSAVE done, 457 keys saved, 0 keys skipped, 118299 bytes written. gitlab # [ 309.282899] redis[3308]: DB saved on disk gitlab # [ 309.285238] redis[3308]: Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB gitlab # [ 309.368658] redis[793]: Background saving terminated with success gitlab # [ 310.770930] postgres[866]: [866] LOG: checkpoint starting: time gitlab # [ 311.870401] gitaly[892]: time="2026-09-23T19:22:42.665Z" level=info msg="burden monitor stats" pid=892 gitlab # [ 317.956091] postgres[3326]: [3326] ERROR: relation "table_without_models" does not exist gitlab # [ 317.956456] postgres[3326]: [3326] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 323.803492] runuser[2983]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 323.806113] setup-gitlab-start[2966]: + systemctl restart gitlab-sidekiq.service gitlab # [ 323.854822] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 323.857189] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:54.650Z","message":"Shutting down"} gitlab # [ 323.859584] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:54.651Z","message":"Scheduler exiting..."} gitlab # [ 323.862205] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:54.652Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 323.865177] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:54.653Z","message":"Scheduler exiting..."} gitlab # [ 324.359544] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:55.155Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 324.863601] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:55.659Z","message":"Bye!"} gitlab # [ 324.903822] sidekiq-cluster[2024]: {"severity":"INFO","time":"2026-09-23T19:22:55.700Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2024,"worker_id":"sidekiq_0","memwd_rss_bytes":966549504,"retry":0} gitlab # [ 326.008275] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 326.008753] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 326.008974] systemd[1]: gitlab-sidekiq.service: Consumed 1min 53.286s CPU time over 3min 22.268s wall clock time, 950.6M memory peak, 4.5M read from disk. gitlab # [ 326.022532] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 326.029250] setup-gitlab-start[2966]: + mkdir -p /var/lib/secrets gitlab # [ 326.037201] setup-gitlab-start[2966]: + echo nixbot-test-token-of-20-chars gitlab # [ 326.040236] setup-gitlab-start[2966]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 326.043438] setup-gitlab-start[2966]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 326.728727] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:22:57 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 638 gitlab # [ 327.921700] gitaly[892]: time="2026-09-23T19:22:58.716Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:08.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:58.714" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.944119] gitaly[892]: time="2026-09-23T19:22:58.716Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:08.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:58.714" grpc.time_ms=1.739 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.171627] gitaly[892]: time="2026-09-23T19:22:58.968Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:08.257" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:58.967" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.212821] gitaly[892]: time="2026-09-23T19:22:58.968Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:08.257" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:58.967" grpc.time_ms=0.631 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.273839] gitaly[892]: time="2026-09-23T19:22:59.070Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 diskcache=b630138d-e03c-479e-b330-e526005b17a6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-23T19:23:08.297" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.008" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.301206] gitaly[892]: time="2026-09-23T19:22:59.071Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=35 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1442 command.oublock=32 command.real_time_ms=38 command.spawn_token_fork_ms=4 command.system_time_ms=35 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-23T19:23:08.297" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.008" grpc.time_ms=62.659 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.325846] gitaly[892]: time="2026-09-23T19:22:59.092Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:09.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.091" pid=892 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 328.341026] gitaly[892]: time="2026-09-23T19:22:59.092Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:09.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.091" grpc.time_ms=1.06 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.356408] gitaly[892]: time="2026-09-23T19:22:59.118Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:23:09.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.100" grpc.time_ms=17.753 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.412635] gitaly[892]: time="2026-09-23T19:22:59.209Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:09.118" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.209" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.438002] gitaly[892]: time="2026-09-23T19:22:59.209Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:09.118" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.209" grpc.time_ms=0.299 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.472418] gitaly[892]: time="2026-09-23T19:22:59.269Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 diskcache=8f9370d7-af28-4078-a575-950e422e66b2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-23T19:23:09.121" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.212" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.490084] gitaly[892]: time="2026-09-23T19:22:59.269Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=33 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1440 command.oublock=32 command.real_time_ms=38 command.spawn_token_fork_ms=6 command.system_time_ms=28 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-23T19:23:09.121" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.212" grpc.time_ms=57.282 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.509842] gitaly[892]: time="2026-09-23T19:22:59.284Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:09.193" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.284" pid=892 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 328.524551] gitaly[892]: time="2026-09-23T19:22:59.285Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:09.193" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.284" grpc.time_ms=0.86 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.537819] gitaly[892]: time="2026-09-23T19:22:59.313Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:23:09.206" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.297" grpc.time_ms=16.3 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.578067] gitaly[892]: time="2026-09-23T19:22:59.374Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:23:09.283" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.374" grpc.time_ms=0.361 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.594660] gitaly[892]: time="2026-09-23T19:22:59.391Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-23T19:23:09.298" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:22:59.389" grpc.time_ms=0.807 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 329.597237] gitaly[892]: time="2026-09-23T19:23:00.393Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M37VH74REKXB597WZZRJ8GJ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:23:10.285" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:23:00.376" grpc.time_ms=17.245 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 331.281779] sidekiq-cluster[3385]: {"severity":"INFO","time":"2026-09-23T19:23:02.036Z","message":"Starting cluster with 1 processes"} gitlab # [ 331.813144] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:02 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 5049 gitlab # [ 331.817099] setup-gitlab-start[3402]: {"id":1,"message":null,"error":null} gitlab # [ 337.489718] gitaly[892]: time="2026-09-23T19:23:08.286Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHH07Q43MGA4M61YM4499 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:23:18.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:23:08.267" grpc.time_ms=18.123 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 338.323239] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:09 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1463 gitlab # [ 338.327605] setup-gitlab-start[2966]: + mkdir -p /tmp/test-flake gitlab # [ 338.332952] setup-gitlab-start[2966]: + cd /tmp/test-flake gitlab # [ 338.338258] setup-gitlab-start[2966]: + git init -b master gitlab # [ 338.361495] setup-gitlab-start[3500]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 338.365152] setup-gitlab-start[2966]: + git config user.name test gitlab # [ 338.368688] setup-gitlab-start[2966]: + git config user.email test@example.com gitlab # [ 338.375466] setup-gitlab-start[2966]: + cat gitlab # [ 338.380943] setup-gitlab-start[2966]: + git add flake.nix gitlab # [ 338.389153] setup-gitlab-start[2966]: + git commit -m 'initial commit' gitlab # [ 338.414104] setup-gitlab-start[3505]: [master (root-commit) c1c83d5] initial commit gitlab # [ 338.419338] setup-gitlab-start[3505]: 1 file changed, 10 insertions(+) gitlab # [ 338.422171] setup-gitlab-start[3505]: create mode 100644 flake.nix gitlab # [ 338.424555] setup-gitlab-start[2966]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 338.430720] setup-gitlab-start[2966]: + git push -u origin master gitlab # [ 338.563709] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:09 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 86 gitlab # [ 338.789795] gitaly[892]: time="2026-09-23T19:23:09.585Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJNJB08NV11HQMEQP9NQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:09.565" grpc.time_ms=19.851 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 338.830817] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:09 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 224 gitlab # [ 339.182731] gitaly[892]: time="2026-09-23T19:23:09.979Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:23:19.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:23:09.960" grpc.time_ms=18.313 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.305304] gitaly[892]: time="2026-09-23T19:23:10.102Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-23T19:23:40.081" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-23T19:23:10.082" grpc.time_ms=19.361 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.368936] gitaly[892]: time="2026-09-23T19:23:10.165Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M37VHJX37KHN0WW0ANV80CSP duration_ms=338ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 339.461201] gitaly[892]: time="2026-09-23T19:23:10.257Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M37VHJX37KHN0WW0ANV80CSP duration_ms=82ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 339.470389] gitaly[892]: time="2026-09-23T19:23:10.258Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:23:09.823" grpc.time_ms=434.945 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 339.519721] gitaly[892]: time="2026-09-23T19:23:10.316Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:23:10.314" grpc.time_ms=2.003 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 339.570760] gitaly[892]: time="2026-09-23T19:23:10.367Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:23:10.366" grpc.time_ms=1.208 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 339.618222] gitaly[892]: time="2026-09-23T19:23:10.415Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:23:10.413" grpc.time_ms=1.174 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 339.837942] gitaly[892]: time="2026-09-23T19:23:10.633Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:23:20.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:23:10.615" grpc.time_ms=17.544 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.878154] gitaly[892]: time="2026-09-23T19:23:10.672Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=5 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-23T19:23:20.033" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:10.644" grpc.time_ms=27.857 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.917426] gitaly[892]: time="2026-09-23T19:23:10.687Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M37VHJX37KHN0WW0ANV80CSP duration_ms=224ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 339.925108] gitaly[892]: time="2026-09-23T19:23:10.689Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:23:10.461" grpc.time_ms=227.227 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 339.947931] gitaly[892]: time="2026-09-23T19:23:10.707Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP diskcache=d39bda6d-38c8-4cd7-a8d7-060895b44361 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:09.704" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 339.967851] setup-gitlab-start[3509]: To http://gitlab/root/test-flake.git gitlab # [ 339.969759] setup-gitlab-start[3509]: * [new branch] master -> master gitlab # [ 339.971401] setup-gitlab-start[3509]: branch 'master' set up to track 'origin/master'. gitlab # [ 339.973986] gitlab-workhorse[819]: time="2026-09-23T19:23:10Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 339.977752] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:10 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1105 gitlab # [ 339.981501] systemd[1]: Finished setup-gitlab.service. gitlab # [ 339.982785] gitaly[892]: time="2026-09-23T19:23:10.714Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=333 command.inblock=8 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13529 command.oublock=32 command.real_time_ms=994 command.spawn_token_fork_ms=2 command.system_time_ms=230 command.user_time_ms=103 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:09.704" grpc.time_ms=1003.818 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 340.005058] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 340.66 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 341.347014] nixbot[3587]: {"ts": "2026-09-23T19:23:12.142Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 341.393960] nixbot[3587]: {"ts": "2026-09-23T19:23:12.190Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 341.436179] nixbot[3587]: {"ts": "2026-09-23T19:23:12.232Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 341.444561] nixbot[3587]: {"ts": "2026-09-23T19:23:12.241Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 341.452311] nixbot[3587]: {"ts": "2026-09-23T19:23:12.249Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 341.464151] nixbot[3587]: {"ts": "2026-09-23T19:23:12.260Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 341.471065] nixbot[3587]: {"ts": "2026-09-23T19:23:12.267Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 341.477874] nixbot[3587]: {"ts": "2026-09-23T19:23:12.271Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 341.486165] nixbot[3587]: {"ts": "2026-09-23T19:23:12.282Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 341.492784] nixbot[3587]: {"ts": "2026-09-23T19:23:12.286Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 341.499947] nixbot[3587]: {"ts": "2026-09-23T19:23:12.296Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 341.511255] nixbot[3587]: {"ts": "2026-09-23T19:23:12.307Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 341.518014] nixbot[3587]: {"ts": "2026-09-23T19:23:12.314Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 341.524836] nixbot[3587]: {"ts": "2026-09-23T19:23:12.318Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 341.531751] nixbot[3587]: {"ts": "2026-09-23T19:23:12.325Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 341.538255] nixbot[3587]: {"ts": "2026-09-23T19:23:12.328Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 341.544088] nixbot[3587]: {"ts": "2026-09-23T19:23:12.331Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 341.549679] nixbot[3587]: {"ts": "2026-09-23T19:23:12.334Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 341.555020] nixbot[3587]: {"ts": "2026-09-23T19:23:12.342Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 341.560393] nixbot[3587]: {"ts": "2026-09-23T19:23:12.346Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 341.565490] nixbot[3587]: {"ts": "2026-09-23T19:23:12.349Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 341.570688] nixbot[3587]: {"ts": "2026-09-23T19:23:12.357Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 341.575941] nixbot[3587]: {"ts": "2026-09-23T19:23:12.360Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 341.581067] nixbot[3587]: {"ts": "2026-09-23T19:23:12.364Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 341.585630] nixbot[3587]: {"ts": "2026-09-23T19:23:12.368Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 341.590195] nixbot[3587]: {"ts": "2026-09-23T19:23:12.372Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 341.594678] nixbot[3587]: {"ts": "2026-09-23T19:23:12.376Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 341.598850] nixbot[3587]: {"ts": "2026-09-23T19:23:12.378Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 341.602671] nixbot[3587]: {"ts": "2026-09-23T19:23:12.381Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 341.606345] nixbot[3587]: {"ts": "2026-09-23T19:23:12.402Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 341.613036] nixbot[3587]: {"ts": "2026-09-23T19:23:12.409Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 341.617111] nixbot[3587]: {"ts": "2026-09-23T19:23:12.412Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 341.621276] nixbot[3587]: {"ts": "2026-09-23T19:23:12.414Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 341.625196] nixbot[3587]: {"ts": "2026-09-23T19:23:12.421Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 341.629165] nixbot[3587]: {"ts": "2026-09-23T19:23:12.424Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 341.639240] nixbot[3587]: {"ts": "2026-09-23T19:23:12.436Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 341.644940] nixbot[3587]: {"ts": "2026-09-23T19:23:12.441Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 341.652349] nixbot[3587]: {"ts": "2026-09-23T19:23:12.449Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "early_attribute_reporting"} gitlab # [ 341.668403] nixbot[3587]: {"ts": "2026-09-23T19:23:12.465Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "terminal_report_reconciliation"} gitlab # [ 342.239045] nixbot[3587]: INFO: Started server process [3587] gitlab # [ 342.242867] nixbot[3587]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 342.268139] nixbot[3587]: INFO: ::ffff:127.0.0.1:42016 - "GET /health HTTP/1.1" 200 OK gitlab # [ 342.275318] systemd[1]: Started nixbot CI service. gitlab # [ 342.277898] systemd[1]: Reached target Multi-User System. gitlab # [ 342.280907] systemd[1]: Startup finished in 855ms (kernel) + 4.106s (initrd) + 5min 37.318s (userspace) = 5min 42.280s. gitlab: (finished: waiting for unit nixbot.service, in 2.15 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 342.391054] nixbot[3587]: INFO: ::ffff:127.0.0.1:32806 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.13 seconds) gitlab # [ 342.518250] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 84 gitlab # [ 342.639815] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:13 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 378 gitlab # [ 342.642961] nixbot[3587]: {"ts": "2026-09-23T19:23:13.439Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 342.688211] nixbot[3587]: {"ts": "2026-09-23T19:23:13.484Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 342.793931] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 92 gitlab # [ 342.797067] nixbot[3587]: {"ts": "2026-09-23T19:23:13.593Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 342.800214] nixbot[3587]: {"ts": "2026-09-23T19:23:13.596Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 342.943163] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 137 gitlab # [ 342.950004] nixbot[3587]: {"ts": "2026-09-23T19:23:13.741Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 343.288916] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:14 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 327 gitlab # [ 343.296354] nixbot[3587]: {"ts": "2026-09-23T19:23:14.088Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 343.418837] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 111 gitlab # [ 343.436410] nixbot[3587]: {"ts": "2026-09-23T19:23:14.224Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 343.542523] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:14 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 104 gitlab # [ 343.558073] nixbot[3587]: {"ts": "2026-09-23T19:23:14.347Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 343.662647] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 83 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.15 seconds) (finished: subtest: project discovered and webhook registered, in 1.28 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 343.706828] gitaly[892]: time="2026-09-23T19:23:14.502Z" level=info msg="finished unary call with code OK" catfile.duration_ms=5 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=5 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=7 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHQHTFYCK991SY5XN9RC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-23T19:23:44.474" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:14.475" grpc.time_ms=26.246 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 343.766277] gitaly[892]: time="2026-09-23T19:23:14.563Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHQHTFYCK991SY5XN9RC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:23:44.560" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:23:14.561" grpc.time_ms=1.334 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 343.813479] gitaly[892]: time="2026-09-23T19:23:14.610Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHQHTFYCK991SY5XN9RC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-23T19:23:24.003" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:23:14.594" grpc.time_ms=15.614 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 343.873952] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 307 gitlab # [ 343.880892] nixbot[3587]: {"ts": "2026-09-23T19:23:14.672Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 344.022035] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 138 gitlab # [ 344.029804] nixbot[3587]: {"ts": "2026-09-23T19:23:14.821Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 344.037898] nixbot[3587]: {"ts": "2026-09-23T19:23:14.828Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 344.235078] gitaly[892]: time="2026-09-23T19:23:15.031Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHR2PQF7C7FKXEEC01R5F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.013" grpc.time_ms=18.662 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.267120] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 130 gitlab # [ 344.424086] gitaly[892]: time="2026-09-23T19:23:15.220Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHR6VG51W37863RB5T2EJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.210" pid=892 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 344.435431] gitaly[892]: time="2026-09-23T19:23:15.221Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHR6VG51W37863RB5T2EJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.210" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.450105] gitlab-workhorse[819]: time="2026-09-23T19:23:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 344.452205] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 187 gitlab # [ 344.454182] gitaly[892]: time="2026-09-23T19:23:15.221Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHR6VG51W37863RB5T2EJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.210" grpc.time_ms=11.306 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.555581] gitaly[892]: time="2026-09-23T19:23:15.352Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRCXYPKD82GKZP2G1BZV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.323" pid=892 remote_ip="2001:db8:1::1" request_sha=8e92026f5be499e19ea16d97a518667c491cdf7f response_bytes=223 span.kind=server system=grpc gitlab # [ 344.580662] gitaly[892]: time="2026-09-23T19:23:15.352Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=1 command.system_time_ms=17 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRCXYPKD82GKZP2G1BZV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.323" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.609404] gitlab-workhorse[819]: time="2026-09-23T19:23:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 344.612844] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 123 gitlab # [ 344.616252] gitaly[892]: time="2026-09-23T19:23:15.352Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=1 command.system_time_ms=17 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRCXYPKD82GKZP2G1BZV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.323" grpc.time_ms=29.494 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.750606] gitaly[892]: time="2026-09-23T19:23:15.544Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHRKBVNF21SJ5BE9CB4NQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.524" grpc.time_ms=19.618 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.787837] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 109 gitlab # [ 344.856886] gitaly[892]: time="2026-09-23T19:23:15.653Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRPXR6FBZEQV2ADB78FQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.640" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 344.869592] gitaly[892]: time="2026-09-23T19:23:15.653Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRPXR6FBZEQV2ADB78FQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.640" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.884571] gitlab-workhorse[819]: time="2026-09-23T19:23:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 344.886500] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 105 gitlab # [ 344.888498] gitaly[892]: time="2026-09-23T19:23:15.654Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRPXR6FBZEQV2ADB78FQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.640" grpc.time_ms=13.645 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.968641] gitaly[892]: time="2026-09-23T19:23:15.765Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VHRVTRQCQRZHAB7KTWKEV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.755" grpc.time_ms=9.775 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 344.982387] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 62 gitlab # [ 345.095945] gitaly[892]: time="2026-09-23T19:23:15.892Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRXWMHXGE8BHT6CA59E9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.870" pid=892 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 345.127973] gitaly[892]: time="2026-09-23T19:23:15.893Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1105 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRXWMHXGE8BHT6CA59E9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.870" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 345.167614] gitlab-workhorse[819]: time="2026-09-23T19:23:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 345.172607] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 121 gitlab # [ 345.177923] gitaly[892]: time="2026-09-23T19:23:15.893Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1105 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHRXWMHXGE8BHT6CA59E9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:15.870" grpc.time_ms=23.192 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 345.388379] gitaly[892]: time="2026-09-23T19:23:16.185Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:23:46.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:23:16.183" grpc.time_ms=1.289 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.454966] gitaly[892]: time="2026-09-23T19:23:16.251Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-23T19:23:46.131" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:16.232" grpc.time_ms=18.883 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.497064] gitaly[892]: time="2026-09-23T19:23:16.285Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:23:26.181" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:16.272" grpc.time_ms=12.831 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.530445] gitaly[892]: time="2026-09-23T19:23:16.314Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:23:26.206" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:16.296" grpc.time_ms=18.041 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.614324] gitaly[892]: time="2026-09-23T19:23:16.411Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-23T19:23:26.009" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:23:16.410" grpc.time_ms=0.592 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.646611] gitaly[892]: time="2026-09-23T19:23:16.436Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-23T19:23:26.021" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:16.421" grpc.time_ms=14.594 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.700177] gitaly[892]: time="2026-09-23T19:23:16.497Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4151 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:23:16.496" grpc.time_ms=0.517 pid=892 span.kind=server system=grpc gitlab # [ 345.743166] gitaly[892]: time="2026-09-23T19:23:16.540Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:23:16.539" grpc.time_ms=0.556 pid=892 span.kind=server system=grpc gitlab # [ 345.767492] gitaly[892]: time="2026-09-23T19:23:16.551Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG diskcache=31e28b82-04b6-4f77-ab2d-4b63a1f0ac0b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:23:26.039" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:23:16.439" pid=892 ref=refs/keep-around/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d remote_ip="2001:db8:1::1" revision=c1c83d50a08750d91e1c12fa878a4e8a8ab3140d span.kind=server system=grpc username=root gitlab # [ 345.789907] gitaly[892]: time="2026-09-23T19:23:16.551Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=111 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5031 command.oublock=8 command.real_time_ms=105 command.spawn_token_fork_ms=1 command.system_time_ms=50 command.user_time_ms=61 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:23:26.039" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:23:16.439" grpc.time_ms=111.369 pid=892 ref=refs/keep-around/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d remote_ip="2001:db8:1::1" revision=c1c83d50a08750d91e1c12fa878a4e8a8ab3140d span.kind=server system=grpc username=root gitlab # [ 345.816978] gitaly[892]: time="2026-09-23T19:23:16.556Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:23:46.455" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:23:16.555" grpc.time_ms=0.658 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.987884] nixbot[3587]: {"ts": "2026-09-23T19:23:16.783Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab # [ 345.993577] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 839 gitlab # [ 346.007047] nixbot[3587]: {"ts": "2026-09-23T19:23:16.803Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-c1c83d50a087-0"} gitlab # [ 346.309244] nixbot[3587]: {"ts": "2026-09-23T19:23:17.105Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-c1c83d50a087-0", "argv0": "bwrap"} gitlab # [ 346.482724] systemd[1]: Started Nix Daemon. gitlab # [ 346.541475] nix-daemon[3757]: accepted connection from pid 3748, user nixbot gitlab # [ 350.856387] nix-daemon[3757]: accepted connection from pid 3796, user nixbot gitlab # [ 350.884950] nix-daemon[3757]: accepted connection from pid 3748, user nixbot gitlab # [ 355.537564] nix-daemon[3757]: accepted connection from pid 3748, user nixbot gitlab # [ 355.687785] gitaly[892]: time="2026-09-23T19:23:26.482Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ3AH3E0RM8XQGPN948E2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:23:56.480" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:23:26.481" grpc.time_ms=1.337 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 355.882819] gitaly[892]: time="2026-09-23T19:23:26.679Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VJ3AH3E0RM8XQGPN948E2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:23:36.050" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:26.661" grpc.time_ms=18.087 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 356.064234] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 442 gitlab # [ 356.073484] nixbot[3587]: {"ts": "2026-09-23T19:23:26.862Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab # [ 356.173283] gitaly[892]: time="2026-09-23T19:23:26.969Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ3RZP0J3H31E6YVTMG46 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:23:56.068" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:23:26.968" grpc.time_ms=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 356.347636] gitaly[892]: time="2026-09-23T19:23:27.144Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VJ3RZP0J3H31E6YVTMG46 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:23:37.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:27.125" grpc.time_ms=18.638 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 356.421174] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 338 gitlab # [ 356.424132] nixbot[3587]: {"ts": "2026-09-23T19:23:27.220Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab # [ 356.510537] gitaly[892]: time="2026-09-23T19:23:27.306Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ4474PMWWH23710MVVXY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:23:57.304" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:23:27.305" grpc.time_ms=1.078 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 356.566626] gitaly[892]: time="2026-09-23T19:23:27.363Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VJ4474PMWWH23710MVVXY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:23:37.226" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:27.346" grpc.time_ms=16.823 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 356.635405] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 191 gitlab # [ 356.643617] nixbot[3587]: {"ts": "2026-09-23T19:23:27.434Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab # [ 356.715683] gitaly[892]: time="2026-09-23T19:23:27.511Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ4ANMPG9SN5Y1BWX9C3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:23:57.409" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:23:27.510" grpc.time_ms=1.123 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 356.765415] gitaly[892]: time="2026-09-23T19:23:27.562Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VJ4ANMPG9SN5Y1BWX9C3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:23:37.146" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:23:27.547" grpc.time_ms=14.582 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 356.832517] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 183 gitlab # [ 356.841703] nixbot[3587]: {"ts": "2026-09-23T19:23:27.630Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab # [ 357.499328] gitaly[892]: time="2026-09-23T19:23:28.296Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M37VJ51CXDPDAA42AW1N6GR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:28.275" grpc.time_ms=19.985 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 357.542932] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 165 gitlab # [ 357.695822] gitaly[892]: time="2026-09-23T19:23:28.492Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ56NA89HPYEZP70PA3MK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:28.472" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 357.721702] gitaly[892]: time="2026-09-23T19:23:28.492Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ56NA89HPYEZP70PA3MK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:28.472" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 357.748219] gitlab-workhorse[819]: time="2026-09-23T19:23:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 357.751585] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 153 gitlab # [ 357.755211] gitaly[892]: time="2026-09-23T19:23:28.493Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ56NA89HPYEZP70PA3MK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:28.472" grpc.time_ms=20.713 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 357.845091] gitaly[892]: time="2026-09-23T19:23:28.641Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VJ5DN48D3BZ40BGYY2AS2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:28.630" grpc.time_ms=11.549 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 357.863530] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 84 gitlab # [ 357.944778] gitaly[892]: time="2026-09-23T19:23:28.741Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ5GADKYAZSQZT27Q2J1P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:28.722" pid=892 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 357.955326] gitaly[892]: time="2026-09-23T19:23:28.741Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ5GADKYAZSQZT27Q2J1P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:28.722" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 357.968920] gitlab-workhorse[819]: time="2026-09-23T19:23:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 357.970771] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:23:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 91 gitlab # [ 357.972732] gitaly[892]: time="2026-09-23T19:23:28.741Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VJ5GADKYAZSQZT27Q2J1P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:23:28.722" grpc.time_ms=19.104 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 374.611602] sidekiq-cluster[3453]: 2026-09-23T19:23:45.407Z pid=3453 tid=yqd INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3453, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426422272} gitlab # [ 374.900713] sidekiq-cluster[3453]: W, [2026-09-23T19:23:45.697501 #3453] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 375.045779] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:23:45.842Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 378.149390] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:23:48.945Z","message":"GitLab reliable fetch activated!"} gitlab # [ 378.149802] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:23:48.945Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 415.157568] postgres[4316]: [4316] ERROR: relation "table_without_models" does not exist gitlab # [ 415.158182] postgres[4316]: [4316] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 418.492467] sidekiq-cluster[3453]: InitializerConnections Query: SET client_min_messages TO 'warning' gitlab # [ 418.492952] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 418.493085] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 418.493203] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 418.493319] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 418.493435] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 418.493553] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 418.493669] sidekiq-cluster[3453]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 418.494629] sidekiq-cluster[3453]: InitializerConnections Query: SET standard_conforming_strings = on gitlab # [ 418.525109] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 418.527628] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 418.528930] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 418.530227] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 418.531398] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 418.532624] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 418.534129] sidekiq-cluster[3453]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 418.568483] sidekiq-cluster[3453]: InitializerConnections Query: SET intervalstyle = iso_8601 gitlab # [ 418.569640] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 418.571038] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 418.572273] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 418.573559] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 418.574713] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 418.575884] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 418.577474] sidekiq-cluster[3453]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 418.646594] sidekiq-cluster[3453]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname gitlab # [ 418.651443] sidekiq-cluster[3453]: FROM pg_type as t gitlab # [ 418.653297] sidekiq-cluster[3453]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz') gitlab # [ 418.657731] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 418.661374] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 418.664509] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 418.667985] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 418.671012] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 418.674123] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 418.678004] sidekiq-cluster[3453]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 418.683204] sidekiq-cluster[3453]: InitializerConnections Query: SET SESSION timezone TO 'UTC' gitlab # [ 418.686186] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 418.689862] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 418.693009] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 418.696269] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 418.699035] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 418.701736] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 418.704938] sidekiq-cluster[3453]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 418.708091] sidekiq-cluster[3453]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'sidekiq_throttling_middleware' gitlab # [ 418.713832] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 418.716338] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 418.718410] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 418.720612] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 418.722519] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 418.724417] sidekiq-cluster[3453]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 418.726724] sidekiq-cluster[3453]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 420.280416] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:31.075Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 420.280849] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:31.077Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 420.280989] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:31.077Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 420.281112] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:31.077Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 420.284497] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:31.081Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 77.70 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 421.632701] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 170 gitlab # [ 421.842952] gitaly[892]: time="2026-09-23T19:24:32.635Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=2 command.system_time_ms=18 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VM3SD5WXKR0CPW79D6WVN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:32.608" grpc.time_ms=26.03 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # Enumerating objects: 5, done. gitlab # [ 421.882401] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 206 gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 272 bytes | 90.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 422.371182] gitaly[892]: time="2026-09-23T19:24:33.167Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=300 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-23T19:25:03.152" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-23T19:24:33.152" grpc.time_ms=14.74 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.413933] gitaly[892]: time="2026-09-23T19:24:33.208Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M37VM41Y0KDYE5SH42XYSC07 duration_ms=181ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 422.487573] gitaly[892]: time="2026-09-23T19:24:33.284Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M37VM41Y0KDYE5SH42XYSC07 duration_ms=74ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 422.496744] gitaly[892]: time="2026-09-23T19:24:33.284Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:33.025" grpc.time_ms=259.002 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 422.548716] gitaly[892]: time="2026-09-23T19:24:33.345Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:33.343" grpc.time_ms=1.617 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 422.608966] gitaly[892]: time="2026-09-23T19:24:33.405Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:33.402" grpc.time_ms=2.595 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 422.653423] gitaly[892]: time="2026-09-23T19:24:33.450Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:33.449" grpc.time_ms=0.756 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 422.804491] gitaly[892]: time="2026-09-23T19:24:33.601Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M37VM41Y0KDYE5SH42XYSC07 duration_ms=98ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 422.808679] gitaly[892]: time="2026-09-23T19:24:33.602Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:33.502" grpc.time_ms=100.229 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 422.822292] gitaly[892]: time="2026-09-23T19:24:33.616Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 diskcache=b0c8123b-18f9-4a42-bd62-5bb7b7fadbc9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:32.912" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 422.835299] gitlab-workhorse[819]: time="2026-09-23T19:24:33Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # c1c83d5..e6cb306 [ 422.838283] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:33 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 915 gitlab # master -> master[ 422.841522] gitaly[892]: time="2026-09-23T19:24:33.617Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=350 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13771 command.oublock=32 command.real_time_ms=695 command.spawn_token_fork_ms=1 command.system_time_ms=202 command.user_time_ms=148 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=604 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:32.912" grpc.time_ms=704.302 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.50 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.14 seconds) gitlab # [ 422.996669] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:33.791Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"ac818ab6954b17254842fb18","created_at":"2026-09-23T19:22:59.138Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:59.183Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-ac818ab6954b17254842fb18: start","job_status":"start","queue_duration_s":94.608714,"scheduling_latency_s":94.608782} gitlab # [ 423.039030] nixbot[3587]: INFO: ::ffff:127.0.0.1:36024 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab # [ 423.245795] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.041Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"ac818ab6954b17254842fb18","created_at":"2026-09-23T19:22:59.138Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:59.183Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-ac818ab6954b17254842fb18: done: 0.249629 sec","job_status":"done","queue_duration_s":94.608714,"scheduling_latency_s":94.608782,"redis_calls":23,"redis_duration_s":0.007193,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003403,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002117,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001673,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.1271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13115012,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.249629,"completed_at":"2026-09-23T19:24:34.041Z","load_balancing_strategy":"primary","db_duration_s":0.079997,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.343123] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.044Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"d97021f7edaddcf5f86f75bb","created_at":"2026-09-23T19:22:59.196Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:59.203Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-d97021f7edaddcf5f86f75bb: start","job_status":"start","queue_duration_s":94.841324,"scheduling_latency_s":94.841364} gitlab # [ 423.436089] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.232Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"d97021f7edaddcf5f86f75bb","created_at":"2026-09-23T19:22:59.196Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-23T19:22:59.203Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-d97021f7edaddcf5f86f75bb: done: 0.187811 sec","job_status":"done","queue_duration_s":94.841324,"scheduling_latency_s":94.841364,"redis_calls":14,"redis_duration_s":0.004937,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000911,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002529,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001497,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.139086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14588396,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.187811,"completed_at":"2026-09-23T19:24:34.232Z","load_balancing_strategy":"primary","db_duration_s":0.037548,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.532617] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.234Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"05fb7eb244c98016995bbeba","created_at":"2026-09-23T19:23:00.193Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:00.203Z","job_size_bytes":5,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-05fb7eb244c98016995bbeba: start","job_status":"start","queue_duration_s":94.030842,"scheduling_latency_s":94.030875} gitlab # [ 423.726484] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.522Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"05fb7eb244c98016995bbeba","created_at":"2026-09-23T19:23:00.193Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:00.203Z","job_size_bytes":5,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-05fb7eb244c98016995bbeba: done: 0.287979 sec","job_status":"done","queue_duration_s":94.030842,"scheduling_latency_s":94.030875,"redis_calls":12,"redis_duration_s":0.003906,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001008,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002324,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000574,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.236442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039666,"duration_s":0.287979,"completed_at":"2026-09-23T19:24:34.522Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043995,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.830836] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.524Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"47cfe690769145d3712c7b79","created_at":"2026-09-23T19:23:00.245Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:00.252Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-47cfe690769145d3712c7b79: start","job_status":"start","queue_duration_s":94.272326,"scheduling_latency_s":94.272352} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.07 seconds) gitlab # [ 424.033492] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.828Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"47cfe690769145d3712c7b79","created_at":"2026-09-23T19:23:00.245Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:00.252Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-47cfe690769145d3712c7b79: done: 0.303792 sec","job_status":"done","queue_duration_s":94.272326,"scheduling_latency_s":94.272352,"redis_calls":11,"redis_duration_s":0.0034260000000000002,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000839,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001951,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000636,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.255,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.255,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.193467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006362,"duration_s":0.303792,"completed_at":"2026-09-23T19:24:34.828Z","load_balancing_strategy":"primary","db_duration_s":0.11384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.138813] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.831Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"7fdf8dcea82791163dfb0a92","created_at":"2026-09-23T19:23:00.257Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:00.264Z","job_size_bytes":89,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-7fdf8dcea82791163dfb0a92: start","job_status":"start","queue_duration_s":94.567182,"scheduling_latency_s":94.567211} gitlab # [ 424.164155] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.880Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"7fdf8dcea82791163dfb0a92","created_at":"2026-09-23T19:23:00.257Z","trace_propagation_headers":{"sentry-trace":"bf81629f0b2f4b6f90e09500862f613e-f878343f5ce34a06","baggage":"sentry-trace_id=bf81629f0b2f4b6f90e09500862f613e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VH74REKXB597WZZRJ8GJ0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:00.264Z","job_size_bytes":89,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-7fdf8dcea82791163dfb0a92: done: 0.048972 sec","job_status":"done","queue_duration_s":94.567182,"scheduling_latency_s":94.567211,"redis_calls":11,"redis_duration_s":0.004462,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000945,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002957,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035208,"duration_s":0.048972,"completed_at":"2026-09-23T19:24:34.880Z","load_balancing_strategy":"primary","db_duration_s":0.005861,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.213456] nixbot[3587]: INFO: ::ffff:127.0.0.1:36028 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab # [ 424.216570] gitaly[892]: time="2026-09-23T19:24:34.952Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:44.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:34.934" grpc.time_ms=18.292 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.229718] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.884Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 c1c83d50a08750d91e1c12fa878a4e8a8ab3140d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"63e5eb91391756ba3c9118e8","created_at":"2026-09-23T19:23:10.529Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-81d83c01cab04ffd","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:23:10.569Z","job_size_bytes":185,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-63e5eb91391756ba3c9118e8: start","job_status":"start","queue_duration_s":84.314511,"scheduling_latency_s":84.314532} gitlab # [ 424.246420] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:34.903Z","message":"changes: \"0000000000000000000000000000000000000000 c1c83d50a08750d91e1c12fa878a4e8a8ab3140d refs/heads/master\\n\""} gitlab # [ 424.248864] gitaly[892]: time="2026-09-23T19:24:34.972Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:24:44.028" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:34.958" grpc.time_ms=14.003 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.262120] gitaly[892]: time="2026-09-23T19:24:35.050Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-23T19:24:45.006" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:35.036" grpc.time_ms=13.1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.321920] gitaly[892]: time="2026-09-23T19:24:35.118Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3934 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:35.116" grpc.time_ms=1.878 pid=892 span.kind=server system=grpc gitlab # [ 424.352559] gitaly[892]: time="2026-09-23T19:24:35.149Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3936 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:35.148" grpc.time_ms=0.542 pid=892 span.kind=server system=grpc gitlab # [ 424.367891] gitaly[892]: time="2026-09-23T19:24:35.164Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP diskcache=5aa746e3-a63c-4af3-b8ac-b93323477eec grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:24:45.033" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:35.064" pid=892 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 424.381487] gitaly[892]: time="2026-09-23T19:24:35.165Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=104 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4986 command.oublock=8 command.real_time_ms=93 command.spawn_token_fork_ms=1 command.system_time_ms=55 command.user_time_ms=49 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:24:45.033" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:35.064" grpc.time_ms=100.762 pid=892 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 424.397522] gitaly[892]: time="2026-09-23T19:24:35.185Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:24:45.154" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:35.185" grpc.time_ms=0.342 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.418217] gitaly[892]: time="2026-09-23T19:24:35.215Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=861 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-23T19:24:45.160" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:35.191" grpc.time_ms=23.484 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.560523] gitaly[892]: time="2026-09-23T19:24:35.357Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-23T19:25:05.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:35.344" grpc.time_ms=12.964 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.611193] gitaly[892]: time="2026-09-23T19:24:35.408Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-23T19:24:45.006" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:35.407" grpc.time_ms=0.582 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.640603] gitaly[892]: time="2026-09-23T19:24:35.422Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-23T19:24:45.011" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-23T19:24:35.411" grpc.time_ms=11.368 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.667341] gitaly[892]: time="2026-09-23T19:24:35.464Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:45.046" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:35.446" grpc.time_ms=17.315 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.692516] gitaly[892]: time="2026-09-23T19:24:35.489Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:45.072" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:35.472" grpc.time_ms=16.315 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.740608] gitaly[892]: time="2026-09-23T19:24:35.537Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-23T19:25:05.208" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:35.509" grpc.time_ms=27.714 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab # [ 425.165032] nixbot[3587]: INFO: ::ffff:127.0.0.1:36030 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.07 seconds) gitlab # [ 425.319569] gitaly[892]: time="2026-09-23T19:24:36.115Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-23T19:24:46.082" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:36.113" grpc.time_ms=1.761 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.358468] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:36.154Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 c1c83d50a08750d91e1c12fa878a4e8a8ab3140d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"63e5eb91391756ba3c9118e8","created_at":"2026-09-23T19:23:10.529Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-81d83c01cab04ffd","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:23:10.569Z","job_size_bytes":185,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-63e5eb91391756ba3c9118e8: done: 1.270094 sec","job_status":"done","queue_duration_s":84.314511,"scheduling_latency_s":84.314532,"gitaly_calls":13,"gitaly_duration_s":0.30784,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.058548,"redis_read_bytes":2103,"redis_write_bytes":21748,"redis_cache_calls":15,"redis_cache_duration_s":0.005693,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008788,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002161,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7836,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.007004,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.020582,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.01432,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.299,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.342,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.803897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00068772,"exclusive_lock_hold_duration_s":0.038016360000000006,"duration_s":1.270094,"completed_at":"2026-09-23T19:24:36.154Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.157968,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 425.444530] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:36.156Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0d0ef32a7a4f94c7e0d66eb6","created_at":"2026-09-23T19:23:16.562Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-33c8ffbe1fa44a01","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:16.571Z","job_size_bytes":65,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0d0ef32a7a4f94c7e0d66eb6: start","job_status":"start","queue_duration_s":79.584598,"scheduling_latency_s":79.584621} gitlab # [ 425.464947] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:36.261Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0d0ef32a7a4f94c7e0d66eb6","created_at":"2026-09-23T19:23:16.562Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-33c8ffbe1fa44a01","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:16.571Z","job_size_bytes":65,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0d0ef32a7a4f94c7e0d66eb6: done: 0.105271 sec","job_status":"done","queue_duration_s":79.584598,"scheduling_latency_s":79.584621,"redis_calls":16,"redis_duration_s":0.005180000000000001,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002958,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000563,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048712,"duration_s":0.105271,"completed_at":"2026-09-23T19:24:36.261Z","load_balancing_strategy":"primary","db_duration_s":0.036023,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 425.513912] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:36.263Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6c6a81978da6aed15bfe5345","created_at":"2026-09-23T19:23:16.572Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-33c8ffbe1fa44a01","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:16.581Z","job_size_bytes":65,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6c6a81978da6aed15bfe5345: start","job_status":"start","queue_duration_s":79.682178,"scheduling_latency_s":79.682201} gitlab # [ 425.603087] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:36.399Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6c6a81978da6aed15bfe5345","created_at":"2026-09-23T19:23:16.572Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-33c8ffbe1fa44a01","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:16.581Z","job_size_bytes":65,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6c6a81978da6aed15bfe5345: done: 0.135502 sec","job_status":"done","queue_duration_s":79.682178,"scheduling_latency_s":79.682201,"redis_calls":21,"redis_duration_s":0.0066879999999999995,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002838,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002233,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004334,"duration_s":0.135502,"completed_at":"2026-09-23T19:24:36.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.674866] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:36.401Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"b6d83c4329b56cf614e29410","created_at":"2026-09-23T19:23:16.751Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-33c8ffbe1fa44a01","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:16.758Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b6d83c4329b56cf614e29410: start","job_status":"start","queue_duration_s":79.642824,"scheduling_latency_s":79.642848} gitlab # [ 425.974205] gitaly[892]: time="2026-09-23T19:24:36.771Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:36.770" grpc.time_ms=0.83 pid=892 span.kind=server system=grpc gitlab # [ 426.019988] gitaly[892]: time="2026-09-23T19:24:36.816Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:36.815" grpc.time_ms=0.823 pid=892 span.kind=server system=grpc gitlab # [ 426.085698] gitaly[892]: time="2026-09-23T19:24:36.882Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:36.880" grpc.time_ms=1.806 pid=892 span.kind=server system=grpc gitlab # [ 426.114341] gitaly[892]: time="2026-09-23T19:24:36.897Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG diskcache=2e5de9f7-c18d-4d59-ab71-4ad214523f8b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:06.416" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:36.717" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.135296] gitaly[892]: time="2026-09-23T19:24:36.897Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=178 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7583 command.oublock=0 command.real_time_ms=172 command.spawn_token_fork_ms=1 command.system_time_ms=113 command.user_time_ms=65 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:06.416" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:36.717" grpc.time_ms=180.445 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab # [ 426.234732] nixbot[3587]: INFO: ::ffff:127.0.0.1:36038 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.07 seconds) gitlab # [ 426.352969] gitaly[892]: time="2026-09-23T19:24:37.148Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:25:07.046" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:37.146" grpc.time_ms=1.776 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.441570] gitaly[892]: time="2026-09-23T19:24:37.238Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHS393JJYM3ADJCD9YCFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:07.135" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:37.236" grpc.time_ms=1.741 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.489498] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:37.282Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"b6d83c4329b56cf614e29410","created_at":"2026-09-23T19:23:16.751Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-33c8ffbe1fa44a01","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:23:16.758Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b6d83c4329b56cf614e29410: done: 0.881683 sec","job_status":"done","queue_duration_s":79.642824,"scheduling_latency_s":79.642848,"gitaly_calls":3,"gitaly_duration_s":0.196537,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.041962,"redis_read_bytes":455,"redis_write_bytes":27998,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001586,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002424,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.015271,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.00639,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14492,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.009817,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000778,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000707,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003403,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.103,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.034,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.038,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.051,"db_ci_duration_s":0.052,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.553605,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.78813784,"duration_s":0.881683,"completed_at":"2026-09-23T19:24:37.282Z","load_balancing_strategy":"primary","db_duration_s":0.121018,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 426.574359] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:37.287Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","c1c83d50a08750d91e1c12fa878a4e8a8ab3140d e6cb306e04578deaef228fc2a54c36a018a70635 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"297cc747dd0246a51a0d4cc8","created_at":"2026-09-23T19:24:33.571Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-5c2ef8ba5e8b4c3d","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:33.576Z","job_size_bytes":185,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-297cc747dd0246a51a0d4cc8: start","job_status":"start","queue_duration_s":3.710708,"scheduling_latency_s":3.710763} gitlab # [ 426.591700] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:37.312Z","message":"changes: \"c1c83d50a08750d91e1c12fa878a4e8a8ab3140d e6cb306e04578deaef228fc2a54c36a018a70635 refs/heads/master\\n\""} gitlab # [ 426.594226] gitaly[892]: time="2026-09-23T19:24:37.364Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:47.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:37.345" grpc.time_ms=18.104 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.607226] gitaly[892]: time="2026-09-23T19:24:37.391Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:24:47.033" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:37.374" grpc.time_ms=16.731 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.620505] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:37.400Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5a1c1694f3ca481fd2601d66","created_at":"2026-09-23T19:24:37.396Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-780c05f470d640e3","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"ded722ed63c30f47f5f7e15b","job_size_bytes":46,"pid":3453,"job_status":"deduplicated","message":"ProjectCacheWorker JID-5a1c1694f3ca481fd2601d66: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 426.650224] gitaly[892]: time="2026-09-23T19:24:37.447Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-23T19:24:47.088" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:37.428" grpc.time_ms=18.349 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.690637] gitaly[892]: time="2026-09-23T19:24:37.473Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-23T19:25:07.155" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:37.455" grpc.time_ms=17.672 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.721854] gitaly[892]: time="2026-09-23T19:24:37.509Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-23T19:24:47.157" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-23T19:24:37.498" grpc.time_ms=10.643 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.748330] gitaly[892]: time="2026-09-23T19:24:37.544Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:47.187" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:37.527" grpc.time_ms=17.034 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.924851] gitaly[892]: time="2026-09-23T19:24:37.721Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-23T19:24:47.009" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:37.719" grpc.time_ms=1.807 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.963752] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:37.758Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","c1c83d50a08750d91e1c12fa878a4e8a8ab3140d e6cb306e04578deaef228fc2a54c36a018a70635 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"297cc747dd0246a51a0d4cc8","created_at":"2026-09-23T19:24:33.571Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-5c2ef8ba5e8b4c3d","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:33.576Z","job_size_bytes":185,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-297cc747dd0246a51a0d4cc8: done: 0.470431 sec","job_status":"done","queue_duration_s":3.710708,"scheduling_latency_s":3.710763,"gitaly_calls":7,"gitaly_duration_s":0.132775,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.038798,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.003606,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.0017,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005992,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.011335,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.016165,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.048,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.048,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.291232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00124246,"exclusive_lock_hold_duration_s":0.0346222,"duration_s":0.470431,"completed_at":"2026-09-23T19:24:37.757Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043318,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 427.049413] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:37.761Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ded722ed63c30f47f5f7e15b","created_at":"2026-09-23T19:24:34.977Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:34.999Z","job_size_bytes":46,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ded722ed63c30f47f5f7e15b: start","job_status":"start","queue_duration_s":2.762346,"scheduling_latency_s":2.762404} gitlab # [ 427.083145] gitaly[892]: time="2026-09-23T19:24:37.880Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-24T01:24:37.870" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:37.871" grpc.time_ms=7.982 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.120267] gitaly[892]: time="2026-09-23T19:24:37.917Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-23T19:25:07.797" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:37.897" grpc.time_ms=18.908 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab # [ 427.185084] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:37.980Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ded722ed63c30f47f5f7e15b","created_at":"2026-09-23T19:24:34.977Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:34.999Z","job_size_bytes":46,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ded722ed63c30f47f5f7e15b: done: 0.218868 sec","job_status":"done","queue_duration_s":2.762346,"scheduling_latency_s":2.762404,"gitaly_calls":2,"gitaly_duration_s":0.035664,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.015795,"redis_read_bytes":69,"redis_write_bytes":4884,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002683,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.001368,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003265,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00592,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002559,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.146304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00097204,"duration_s":0.218868,"completed_at":"2026-09-23T19:24:37.980Z","load_balancing_strategy":"primary","db_duration_s":0.021373,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 427.276219] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:37.984Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"017f509d14fa6d75bbfae030","created_at":"2026-09-23T19:24:35.024Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:35.031Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-017f509d14fa6d75bbfae030: start","job_status":"start","queue_duration_s":2.953371,"scheduling_latency_s":2.953419} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.07 seconds) gitlab # [ 427.299294] gitaly[892]: time="2026-09-23T19:24:38.078Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP diskcache=3e4239a4-e19d-4edb-86e0-06a347b0ff88 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-24T01:24:38.037" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:38.038" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.314309] gitaly[892]: time="2026-09-23T19:24:38.079Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=51 command.spawn_token_fork_ms=6 command.system_time_ms=24 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-24T01:24:38.037" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:38.038" grpc.time_ms=40.908 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.334906] nixbot[3587]: INFO: ::ffff:127.0.0.1:36050 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab # [ 427.359481] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:38.155Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"017f509d14fa6d75bbfae030","created_at":"2026-09-23T19:24:35.024Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:35.031Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-017f509d14fa6d75bbfae030: done: 0.171059 sec","job_status":"done","queue_duration_s":2.953371,"scheduling_latency_s":2.953419,"gitaly_calls":1,"gitaly_duration_s":0.045397,"redis_calls":15,"redis_duration_s":0.006197,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000747,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001191,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.038,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11801822000000001,"duration_s":0.171059,"completed_at":"2026-09-23T19:24:38.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.419854] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:38.158Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"5e72fcfee8abeabceb097d0a","created_at":"2026-09-23T19:24:35.178Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:35.181Z","job_size_bytes":23,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-5e72fcfee8abeabceb097d0a: start","job_status":"start","queue_duration_s":2.977323,"scheduling_latency_s":2.977362} gitlab # [ 427.435594] gitaly[892]: time="2026-09-23T19:24:38.226Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-23T19:25:08.106" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:38.206" grpc.time_ms=18.914 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.448523] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:38.240Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"5e72fcfee8abeabceb097d0a","created_at":"2026-09-23T19:24:35.178Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:35.181Z","job_size_bytes":23,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-5e72fcfee8abeabceb097d0a: done: 0.081679 sec","job_status":"done","queue_duration_s":2.977323,"scheduling_latency_s":2.977362,"gitaly_calls":1,"gitaly_duration_s":0.022332,"redis_calls":16,"redis_duration_s":0.005223,"redis_read_bytes":52,"redis_write_bytes":2045,"redis_queues_calls":1,"redis_queues_duration_s":0.000423,"redis_queues_read_bytes":1,"redis_queues_write_bytes":976,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002264,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.081679,"completed_at":"2026-09-23T19:24:38.240Z","load_balancing_strategy":"primary","db_duration_s":0.005372,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 427.492493] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:38.243Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"29618c8619c14392c18549e7","created_at":"2026-09-23T19:24:35.912Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:29b68cfa4c8ad612be1610ebb616c4d4d5d91153d535a9ae92ff5565969523a3","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:35.921Z","job_size_bytes":467,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-29618c8619c14392c18549e7: start","job_status":"start","queue_duration_s":2.322176,"scheduling_latency_s":2.322207} gitlab # [ 427.510343] gitaly[892]: time="2026-09-23T19:24:38.282Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:08.180" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:38.281" grpc.time_ms=1.081 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.522452] gitaly[892]: time="2026-09-23T19:24:38.301Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-23T19:24:48.005" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:38.286" grpc.time_ms=14.603 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.660726] gitaly[892]: time="2026-09-23T19:24:38.457Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:25:08.354" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:38.455" grpc.time_ms=1.895 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.698679] gitaly[892]: time="2026-09-23T19:24:38.465Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=33 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-23T19:25:08.361" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:38.461" grpc.time_ms=4.153 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.785271] gitaly[892]: time="2026-09-23T19:24:38.582Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:48.222" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:38.563" grpc.time_ms=18.693 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.07 seconds) gitlab # [ 428.376916] nixbot[3587]: INFO: ::ffff:127.0.0.1:36052 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab # [ 428.658218] gitaly[892]: time="2026-09-23T19:24:39.453Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=23 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=19 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-23T19:25:09.325" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:39.425" grpc.time_ms=28.056 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.698842] gitaly[892]: time="2026-09-23T19:24:39.486Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VHJX37KHN0WW0ANV80CSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-23T19:25:09.357" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:39.457" grpc.time_ms=28.122 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.738561] sidekiq-cluster[3453]: {"severity":"WARN","time":"2026-09-23T19:24:39.534Z","class":"CreatePipelineWorker","correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"c1c83d50a08750d91e1c12fa878a4e8a8ab3140d","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"c1c83d50a08750d91e1c12fa878a4e8a8ab3140d","ref":"refs/heads/master"},"retry":0} gitlab # [ 428.759978] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:39.546Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"29618c8619c14392c18549e7","created_at":"2026-09-23T19:24:35.912Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:29b68cfa4c8ad612be1610ebb616c4d4d5d91153d535a9ae92ff5565969523a3","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:35.921Z","job_size_bytes":467,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-29618c8619c14392c18549e7: done: 1.302558 sec","job_status":"done","queue_duration_s":2.322176,"scheduling_latency_s":2.322207,"gitaly_calls":7,"gitaly_duration_s":0.119088,"redis_calls":51,"redis_duration_s":0.014636,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001585,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.006032,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001899,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003723,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001397,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.098335,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063388,"duration_s":1.302558,"completed_at":"2026-09-23T19:24:39.545Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.087052,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.849808] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:39.548Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"8c85b8c1-b2e2-42ca-940e-5df06a623efb\"}"],"class":"WebHookWorker","jid":"0eea1fbf518212938e0b07d7","created_at":"2026-09-23T19:24:36.072Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dd7424cdbfb2e94b32fda5962cb37ce9290789fd01976b6a6c594406fa4fc116","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.078Z","job_size_bytes":1751,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0eea1fbf518212938e0b07d7: start","job_status":"start","queue_duration_s":3.469956,"scheduling_latency_s":3.469989} gitlab # [ 428.961839] nixbot[3587]: INFO: ::ffff:127.0.0.1:36064 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 429.006919] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:39.802Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"8c85b8c1-b2e2-42ca-940e-5df06a623efb\"}"],"class":"WebHookWorker","jid":"0eea1fbf518212938e0b07d7","created_at":"2026-09-23T19:24:36.072Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-1b61200869204c5f","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dd7424cdbfb2e94b32fda5962cb37ce9290789fd01976b6a6c594406fa4fc116","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.078Z","job_size_bytes":1751,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0eea1fbf518212938e0b07d7: done: 0.253629 sec","job_status":"done","queue_duration_s":3.469956,"scheduling_latency_s":3.469989,"redis_calls":19,"redis_duration_s":0.007446,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001955,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000717,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003022,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001752,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.10368909999999687,"cpu_s":0.127072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0009567,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.253629,"completed_at":"2026-09-23T19:24:39.802Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.084166] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:39.808Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7ce995d1fa2a52097316fb3e","created_at":"2026-09-23T19:24:36.698Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.704Z","job_size_bytes":23,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7ce995d1fa2a52097316fb3e: start","job_status":"start","queue_duration_s":3.104056,"scheduling_latency_s":3.104112} gitlab # [ 429.122616] gitaly[892]: time="2026-09-23T19:24:39.919Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VMB03YDGHVMQNFQWFG85Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:39.901" grpc.time_ms=17.375 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.145279] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 109 gitlab # [ 429.162281] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:39.958Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7ce995d1fa2a52097316fb3e","created_at":"2026-09-23T19:24:36.698Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.704Z","job_size_bytes":23,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7ce995d1fa2a52097316fb3e: done: 0.150115 sec","job_status":"done","queue_duration_s":3.104056,"scheduling_latency_s":3.104112,"redis_calls":10,"redis_duration_s":0.003194,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000686,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001921,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000587,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037826,"duration_s":0.150115,"completed_at":"2026-09-23T19:24:39.958Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029516,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 429.221666] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:39.960Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"3e9999f70044b1b0b50ef7f1","created_at":"2026-09-23T19:24:36.906Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.918Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-3e9999f70044b1b0b50ef7f1: start","job_status":"start","queue_duration_s":3.041965,"scheduling_latency_s":3.041993} gitlab # [ 429.238534] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.023Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"3e9999f70044b1b0b50ef7f1","created_at":"2026-09-23T19:24:36.906Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.918Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-3e9999f70044b1b0b50ef7f1: done: 0.062487 sec","job_status":"done","queue_duration_s":3.041965,"scheduling_latency_s":3.041993,"redis_calls":10,"redis_duration_s":0.0022010000000000003,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039478,"duration_s":0.062487,"completed_at":"2026-09-23T19:24:40.022Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012433,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.282658] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.024Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2d44a9314b21e46f0028265e","created_at":"2026-09-23T19:24:36.945Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.958Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2d44a9314b21e46f0028265e: start","job_status":"start","queue_duration_s":3.066349,"scheduling_latency_s":3.066372} gitlab # [ 429.298366] gitlab-workhorse[819]: time="2026-09-23T19:24:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 429.300214] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 110 gitlab # [ 429.302070] gitaly[892]: time="2026-09-23T19:24:40.032Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMB3KQQYSJVHJSZCXYA1M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:40.014" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 429.312172] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.078Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2d44a9314b21e46f0028265e","created_at":"2026-09-23T19:24:36.945Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.958Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2d44a9314b21e46f0028265e: done: 0.05363 sec","job_status":"done","queue_duration_s":3.066349,"scheduling_latency_s":3.066372,"redis_calls":10,"redis_duration_s":0.003773,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00293,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00172966,"duration_s":0.05363,"completed_at":"2026-09-23T19:24:40.078Z","load_balancing_strategy":"primary","db_duration_s":0.010073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.355972] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.080Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"7aa93bd973885a12d88dcce7","created_at":"2026-09-23T19:24:36.985Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.997Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7aa93bd973885a12d88dcce7: start","job_status":"start","queue_duration_s":3.083034,"scheduling_latency_s":3.083057} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab # [ 429.372482] gitaly[892]: time="2026-09-23T19:24:40.032Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMB3KQQYSJVHJSZCXYA1M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:40.014" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.385354] gitlab-workhorse[819]: time="2026-09-23T19:24:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=336 gitlab # [ 429.387041] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 89 gitlab # [ 429.388860] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"7aa93bd973885a12d88dcce7","created_at":"2026-09-23T19:24:36.985Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:36.997Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7aa93bd973885a12d88dcce7: done: 0.052251 sec","job_status":"done","queue_duration_s":3.083034,"scheduling_latency_s":3.083057,"redis_calls":13,"redis_duration_s":0.003757,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000788,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000362,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028706,"duration_s":0.052251,"completed_at":"2026-09-23T19:24:40.133Z","load_balancing_strategy":"primary","db_duration_s":0.009449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.434578] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.135Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"cc81bca45521a63bdc2a10dc","created_at":"2026-09-23T19:24:36.998Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.011Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-cc81bca45521a63bdc2a10dc: start","job_status":"start","queue_duration_s":3.123633,"scheduling_latency_s":3.123655} gitlab # [ 429.451263] gitaly[892]: time="2026-09-23T19:24:40.032Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMB3KQQYSJVHJSZCXYA1M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:40.014" grpc.time_ms=18.557 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.466248] nixbot[3587]: {"ts": "2026-09-23T19:24:40.183Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 429.468739] nixbot[3587]: INFO: ::ffff:127.0.0.1:36076 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.09 seconds) gitlab # [ 429.470876] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.178Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"cc81bca45521a63bdc2a10dc","created_at":"2026-09-23T19:24:36.998Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.011Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-cc81bca45521a63bdc2a10dc: done: 0.043948 sec","job_status":"done","queue_duration_s":3.123633,"scheduling_latency_s":3.123655,"redis_calls":10,"redis_duration_s":0.001443,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024816,"duration_s":0.043948,"completed_at":"2026-09-23T19:24:40.178Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.515288] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d600afcc6ee87c3ec0c227d7","created_at":"2026-09-23T19:24:37.012Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.027Z","job_size_bytes":19,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d600afcc6ee87c3ec0c227d7: start","job_status":"start","queue_duration_s":3.153195,"scheduling_latency_s":3.153217} gitlab # [ 429.531560] gitaly[892]: time="2026-09-23T19:24:40.125Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMB75XQF8AVQFWJEK2H6C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:40.101" pid=892 remote_ip="2001:db8:1::1" request_sha=f4df98610523d1cc3cda1c8943014e6fde48d2af response_bytes=336 span.kind=server system=grpc gitlab # [ 429.541649] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.235Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d600afcc6ee87c3ec0c227d7","created_at":"2026-09-23T19:24:37.012Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.027Z","job_size_bytes":19,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d600afcc6ee87c3ec0c227d7: done: 0.055335 sec","job_status":"done","queue_duration_s":3.153195,"scheduling_latency_s":3.153217,"redis_calls":11,"redis_duration_s":0.00585,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000608,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004635,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041604,"duration_s":0.055335,"completed_at":"2026-09-23T19:24:40.235Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010152,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.585602] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.237Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"25fd91a7a2f8ad1818b51515","created_at":"2026-09-23T19:24:37.034Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.044Z","job_size_bytes":66,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-25fd91a7a2f8ad1818b51515: start","job_status":"start","queue_duration_s":3.192932,"scheduling_latency_s":3.192954} gitlab # [ 429.602179] gitaly[892]: time="2026-09-23T19:24:40.126Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1126 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMB75XQF8AVQFWJEK2H6C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:40.101" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.615774] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.276Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"25fd91a7a2f8ad1818b51515","created_at":"2026-09-23T19:24:37.034Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.044Z","job_size_bytes":66,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-25fd91a7a2f8ad1818b51515: done: 0.038759 sec","job_status":"done","queue_duration_s":3.192932,"scheduling_latency_s":3.192954,"redis_calls":10,"redis_duration_s":0.003045,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000834,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00162,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003499,"duration_s":0.038759,"completed_at":"2026-09-23T19:24:40.276Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004858,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.659948] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.277Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"37ed894b4de8cd57a91f9258","created_at":"2026-09-23T19:24:37.046Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.056Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-37ed894b4de8cd57a91f9258: start","job_status":"start","queue_duration_s":3.221269,"scheduling_latency_s":3.22129} gitlab # [ 429.675943] gitaly[892]: time="2026-09-23T19:24:40.127Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1126 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMB75XQF8AVQFWJEK2H6C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:40.101" grpc.time_ms=25.102 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.690573] nixbot[3587]: {"ts": "2026-09-23T19:24:40.469Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab # [ 429.693614] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.358Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"37ed894b4de8cd57a91f9258","created_at":"2026-09-23T19:24:37.046Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.056Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-37ed894b4de8cd57a91f9258: done: 0.080815 sec","job_status":"done","queue_duration_s":3.221269,"scheduling_latency_s":3.22129,"redis_calls":10,"redis_duration_s":0.003295,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000919,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054424,"duration_s":0.080815,"completed_at":"2026-09-23T19:24:40.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.737174] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.361Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"bbf25222f1db38092d16df12","created_at":"2026-09-23T19:24:37.070Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.078Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-bbf25222f1db38092d16df12: start","job_status":"start","queue_duration_s":3.282602,"scheduling_latency_s":3.282632} gitlab # [ 429.753639] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 261 gitlab # [ 429.756133] gitaly[892]: time="2026-09-23T19:24:40.276Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:10.274" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:40.275" grpc.time_ms=1.041 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.768818] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.411Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"bbf25222f1db38092d16df12","created_at":"2026-09-23T19:24:37.070Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.078Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-bbf25222f1db38092d16df12: done: 0.050988 sec","job_status":"done","queue_duration_s":3.282602,"scheduling_latency_s":3.282632,"redis_calls":10,"redis_duration_s":0.002826,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033322,"duration_s":0.050988,"completed_at":"2026-09-23T19:24:40.411Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.813882] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.413Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"65717647fbc1df7ea32e230a","created_at":"2026-09-23T19:24:37.079Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.087Z","job_size_bytes":6,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-65717647fbc1df7ea32e230a: start","job_status":"start","queue_duration_s":3.326435,"scheduling_latency_s":3.326458} gitlab # [ 429.829719] gitaly[892]: time="2026-09-23T19:24:40.378Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:50.240" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:40.360" grpc.time_ms=17.99 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.842814] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.440Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"65717647fbc1df7ea32e230a","created_at":"2026-09-23T19:24:37.079Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.087Z","job_size_bytes":6,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-65717647fbc1df7ea32e230a: done: 0.027273 sec","job_status":"done","queue_duration_s":3.326435,"scheduling_latency_s":3.326458,"redis_calls":10,"redis_duration_s":0.002009,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002243,"duration_s":0.027273,"completed_at":"2026-09-23T19:24:40.440Z","load_balancing_strategy":"primary","db_duration_s":0.00396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.886071] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.442Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"9838198bfb8315eafef416a3","created_at":"2026-09-23T19:24:37.088Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.097Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-9838198bfb8315eafef416a3: start","job_status":"start","queue_duration_s":3.345112,"scheduling_latency_s":3.34513} gitlab # [ 429.901232] gitaly[892]: time="2026-09-23T19:24:40.581Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMBMXB7G2KJ0ETK5WE09E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:10.479" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:40.579" grpc.time_ms=0.871 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.913810] gitaly[892]: time="2026-09-23T19:24:40.596Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 diskcache=f8b6a0d1-7ca5-49f0-9766-fb00369226f3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-24T01:24:40.582" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:40.582" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.923385] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"9838198bfb8315eafef416a3","created_at":"2026-09-23T19:24:37.088Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.097Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-9838198bfb8315eafef416a3: done: 0.039927 sec","job_status":"done","queue_duration_s":3.345112,"scheduling_latency_s":3.34513,"redis_calls":10,"redis_duration_s":0.00197,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000395,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00051,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034104,"duration_s":0.039927,"completed_at":"2026-09-23T19:24:40.482Z","load_balancing_strategy":"primary","db_duration_s":0.008683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.965945] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.484Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"52cd537c01b8fcd06801da78","created_at":"2026-09-23T19:24:37.110Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.121Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-52cd537c01b8fcd06801da78: start","job_status":"start","queue_duration_s":3.362544,"scheduling_latency_s":3.362565} gitlab # [ 429.981870] gitaly[892]: time="2026-09-23T19:24:40.597Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=291 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-24T01:24:40.582" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:40.582" grpc.time_ms=14.066 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.995909] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 259 gitlab # [ 429.998495] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.559Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"52cd537c01b8fcd06801da78","created_at":"2026-09-23T19:24:37.110Z","trace_propagation_headers":{"sentry-trace":"8aaa3d0a9d82473fa8e8244e1a46fd7c-11df97b74aa54733","baggage":"sentry-trace_id=8aaa3d0a9d82473fa8e8244e1a46fd7c,sentry-release=Unknown"},"correlation_id":"01M37VHS393JJYM3ADJCD9YCFG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.121Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-52cd537c01b8fcd06801da78: done: 0.07524 sec","job_status":"done","queue_duration_s":3.362544,"scheduling_latency_s":3.362565,"redis_calls":14,"redis_duration_s":0.003845,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002205,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001168,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0010248,"duration_s":0.07524,"completed_at":"2026-09-23T19:24:40.559Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.041749] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.560Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a22e1c5d07d511ed50bf7d34","created_at":"2026-09-23T19:24:37.415Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-780c05f470d640e3","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"017f509d14fa6d75bbfae030","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.420Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a22e1c5d07d511ed50bf7d34: start","job_status":"start","queue_duration_s":3.140271,"scheduling_latency_s":3.140292} gitlab # [ 430.058349] nixbot[3587]: {"ts": "2026-09-23T19:24:40.733Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab # [ 430.061267] gitaly[892]: time="2026-09-23T19:24:40.651Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:10.550" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:40.650" grpc.time_ms=0.565 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.074037] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.620Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a22e1c5d07d511ed50bf7d34","created_at":"2026-09-23T19:24:37.415Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-780c05f470d640e3","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"017f509d14fa6d75bbfae030","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.420Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a22e1c5d07d511ed50bf7d34: done: 0.059327 sec","job_status":"done","queue_duration_s":3.140271,"scheduling_latency_s":3.140292,"gitaly_calls":1,"gitaly_duration_s":0.0167,"redis_calls":12,"redis_duration_s":0.005442,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000451,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00329,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03464484,"duration_s":0.059327,"completed_at":"2026-09-23T19:24:40.620Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006544,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.119009] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:40.622Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b134c24260b9d0f33f34c216","created_at":"2026-09-23T19:24:37.654Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-780c05f470d640e3","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:794ff0f5b221641ec32880ba3471009cc4f6023f2a5baa5a37c97a4bdb0594ae","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.659Z","job_size_bytes":467,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b134c24260b9d0f33f34c216: start","job_status":"start","queue_duration_s":2.96228,"scheduling_latency_s":2.962303} gitlab # [ 430.135996] gitaly[892]: time="2026-09-23T19:24:40.669Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VMBMXB7G2KJ0ETK5WE09E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:50.260" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:40.661" grpc.time_ms=8.672 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.149085] gitaly[892]: time="2026-09-23T19:24:40.672Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-23T19:24:50.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:40.654" grpc.time_ms=18.12 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.162290] gitaly[892]: time="2026-09-23T19:24:40.721Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:25:10.018" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:40.719" grpc.time_ms=2.218 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.175248] nixbot[3587]: {"ts": "2026-09-23T19:24:40.947Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab # [ 430.178555] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 190 gitlab # [ 430.181144] gitaly[892]: time="2026-09-23T19:24:40.729Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=23 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-23T19:25:10.026" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:40.726" grpc.time_ms=2.815 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.197272] gitaly[892]: time="2026-09-23T19:24:40.778Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:50.051" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:40.762" grpc.time_ms=16.378 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.211470] gitaly[892]: time="2026-09-23T19:24:40.823Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMBXKGSXV6VM3JX78AG4R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:10.722" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:40.822" grpc.time_ms=0.574 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.224170] gitaly[892]: time="2026-09-23T19:24:40.882Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VMBXKGSXV6VM3JX78AG4R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:50.155" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:40.865" grpc.time_ms=16.726 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.237460] gitaly[892]: time="2026-09-23T19:24:41.014Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMC3TBTSHKYPCHYY9VMP3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:11.013" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:41.013" grpc.time_ms=0.788 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.275612] gitaly[892]: time="2026-09-23T19:24:41.072Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VMC3TBTSHKYPCHYY9VMP3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:51.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:41.059" grpc.time_ms=12.772 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.333909] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:41 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 175 gitlab # [ 430.338753] nixbot[3587]: {"ts": "2026-09-23T19:24:41.131Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c1c83d50a08750d91e1c12fa878a4e8a8ab3140d \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab # [ 430.459782] gitaly[892]: time="2026-09-23T19:24:41.256Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VMCAV2S0B36NQ36KPJGQ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:41.236" grpc.time_ms=19.611 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 430.496077] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:41 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 77 gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.09 seconds) gitlab # [ 430.548129] gitaly[892]: time="2026-09-23T19:24:41.344Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMCDB12VEHGWW55RMNN1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:41.324" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 430.569428] gitaly[892]: time="2026-09-23T19:24:41.344Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMCDB12VEHGWW55RMNN1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:41.324" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 430.592818] nixbot[3587]: INFO: ::ffff:127.0.0.1:36078 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab # [ 430.595652] gitlab-workhorse[819]: time="2026-09-23T19:24:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 430.598414] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:41 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 86 gitlab # [ 430.601081] gitaly[892]: time="2026-09-23T19:24:41.344Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMCDB12VEHGWW55RMNN1Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:41.324" grpc.time_ms=19.742 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 430.719826] gitaly[892]: time="2026-09-23T19:24:41.515Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-23T19:25:11.087" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:41.488" grpc.time_ms=27.086 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.762184] gitaly[892]: time="2026-09-23T19:24:41.538Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VM41Y0KDYE5SH42XYSC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-23T19:25:11.122" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:41.523" grpc.time_ms=14.818 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.786079] sidekiq-cluster[3453]: {"severity":"WARN","time":"2026-09-23T19:24:41.556Z","class":"CreatePipelineWorker","correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"c1c83d50a08750d91e1c12fa878a4e8a8ab3140d","after":"e6cb306e04578deaef228fc2a54c36a018a70635","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"e6cb306e04578deaef228fc2a54c36a018a70635","ref":"refs/heads/master"},"retry":0} gitlab # [ 430.796324] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:41.563Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b134c24260b9d0f33f34c216","created_at":"2026-09-23T19:24:37.654Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-780c05f470d640e3","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:794ff0f5b221641ec32880ba3471009cc4f6023f2a5baa5a37c97a4bdb0594ae","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.659Z","job_size_bytes":467,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b134c24260b9d0f33f34c216: done: 0.941394 sec","job_status":"done","queue_duration_s":2.96228,"scheduling_latency_s":2.962303,"gitaly_calls":7,"gitaly_duration_s":0.103838,"redis_calls":33,"redis_duration_s":0.013896,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.002933,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.006513,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002955,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.803117,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.941394,"completed_at":"2026-09-23T19:24:41.563Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027754,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 430.856929] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:41.566Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e909bc83-d3ff-45a2-85f3-d9410e67c2f8\"}"],"class":"WebHookWorker","jid":"85fd327b0f51aa2aa28a89ce","created_at":"2026-09-23T19:24:37.690Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-780c05f470d640e3","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e0c149ecda04048ff682dfb52f8962d4f1259298e7261470264fad8d7821f35f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.694Z","job_size_bytes":1749,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-85fd327b0f51aa2aa28a89ce: start","job_status":"start","queue_duration_s":3.871725,"scheduling_latency_s":3.871754} gitlab # [ 430.875580] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:41.625Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e909bc83-d3ff-45a2-85f3-d9410e67c2f8\"}"],"class":"WebHookWorker","jid":"85fd327b0f51aa2aa28a89ce","created_at":"2026-09-23T19:24:37.690Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-780c05f470d640e3","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e0c149ecda04048ff682dfb52f8962d4f1259298e7261470264fad8d7821f35f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.694Z","job_size_bytes":1749,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-85fd327b0f51aa2aa28a89ce: done: 0.059056 sec","job_status":"done","queue_duration_s":3.871725,"scheduling_latency_s":3.871754,"redis_calls":14,"redis_duration_s":0.005038,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.001363,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002147,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001528,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.00924129999998513,"cpu_s":0.036298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.059056,"completed_at":"2026-09-23T19:24:41.624Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.922794] nixbot[3587]: INFO: ::ffff:127.0.0.1:36082 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 430.924053] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:41.630Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"cc176a37fd8c01d0c6945844","created_at":"2026-09-23T19:24:37.936Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-77324d1e85c64e32","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.949Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-cc176a37fd8c01d0c6945844: start","job_status":"start","queue_duration_s":3.680769,"scheduling_latency_s":3.680821} gitlab # [ 430.943087] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:41.739Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"cc176a37fd8c01d0c6945844","created_at":"2026-09-23T19:24:37.936Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-77324d1e85c64e32","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:37.949Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-cc176a37fd8c01d0c6945844: done: 0.109418 sec","job_status":"done","queue_duration_s":3.680769,"scheduling_latency_s":3.680821,"redis_calls":19,"redis_duration_s":0.013344,"redis_read_bytes":21,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001342,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.00244,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.007576,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001986,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0172252,"duration_s":0.109418,"completed_at":"2026-09-23T19:24:41.739Z","load_balancing_strategy":"primary","db_duration_s":0.018326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.988610] gitaly[892]: time="2026-09-23T19:24:41.768Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M37VMCSBP2KM179SY6WBZRRQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:41.745" grpc.time_ms=22.588 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.001665] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:41 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 125 gitlab # [ 431.003746] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:41.741Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0eea1fbf518212938e0b07d7"],"class":"WebHooks::LogExecutionWorker","jid":"f8762f965d484cd09af9a079","created_at":"2026-09-23T19:24:39.764Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-58e8c894df8c49ea","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:08ee14303ad7e53e69d69622af5927839a88728ca48203b8de5c5c544471a8ed","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:39.779Z","job_size_bytes":2593,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f8762f965d484cd09af9a079: start","job_status":"start","queue_duration_s":1.961868,"scheduling_latency_s":1.961901} gitlab # [ 431.021798] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:41.816Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0eea1fbf518212938e0b07d7"],"class":"WebHooks::LogExecutionWorker","jid":"f8762f965d484cd09af9a079","created_at":"2026-09-23T19:24:39.764Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-58e8c894df8c49ea","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:08ee14303ad7e53e69d69622af5927839a88728ca48203b8de5c5c544471a8ed","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:39.779Z","job_size_bytes":2593,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f8762f965d484cd09af9a079: done: 0.075181 sec","job_status":"done","queue_duration_s":1.961868,"scheduling_latency_s":1.961901,"redis_calls":10,"redis_duration_s":0.00355,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000876,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001864,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00081,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048572,"duration_s":0.075181,"completed_at":"2026-09-23T19:24:41.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017827,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.066617] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:41.818Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7d8593c892d4f49599958ffd","created_at":"2026-09-23T19:24:40.429Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-303ce4d9d071455c","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:40.441Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7d8593c892d4f49599958ffd: start","job_status":"start","queue_duration_s":1.377161,"scheduling_latency_s":1.377192} gitlab # [ 431.089866] gitaly[892]: time="2026-09-23T19:24:41.886Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMCXBB2HP7FPC3EF3XXGD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:41.876" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 431.100358] gitaly[892]: time="2026-09-23T19:24:41.886Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMCXBB2HP7FPC3EF3XXGD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:41.876" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.113785] gitlab-workhorse[819]: time="2026-09-23T19:24:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 431.115586] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:41 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 116 gitlab # [ 431.117544] gitaly[892]: time="2026-09-23T19:24:41.887Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMCXBB2HP7FPC3EF3XXGD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:41.876" grpc.time_ms=10.655 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.178073] gitaly[892]: time="2026-09-23T19:24:41.975Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:41.974" grpc.time_ms=0.426 pid=892 span.kind=server system=grpc gitlab # [ 431.216653] gitaly[892]: time="2026-09-23T19:24:42.013Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:42.012" grpc.time_ms=1.054 pid=892 span.kind=server system=grpc gitlab # [ 431.230206] gitaly[892]: time="2026-09-23T19:24:42.014Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M37VMD2HW2TNRH03XYHQ0ZEH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:42.004" grpc.time_ms=9.332 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.246420] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:42 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 77 gitlab # [ 431.250573] gitaly[892]: time="2026-09-23T19:24:42.047Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:42.046" grpc.time_ms=1.106 pid=892 span.kind=server system=grpc gitlab # [ 431.261703] gitaly[892]: time="2026-09-23T19:24:42.055Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA diskcache=bed7b51c-d764-4fa9-99c2-66ed880eeb1d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:11.243" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:41.943" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.272229] gitaly[892]: time="2026-09-23T19:24:42.055Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=99 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7269 command.oublock=0 command.real_time_ms=106 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=60 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:11.243" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:41.943" grpc.time_ms=111.796 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.332147] gitaly[892]: time="2026-09-23T19:24:42.129Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMD504AX8BSM0X3RFHTVE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:42.110" pid=892 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 431.348310] gitaly[892]: time="2026-09-23T19:24:42.129Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMD504AX8BSM0X3RFHTVE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:42.110" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.366502] gitlab-workhorse[819]: time="2026-09-23T19:24:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 431.368767] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 113 gitlab # [ 431.371037] gitaly[892]: time="2026-09-23T19:24:42.129Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMD504AX8BSM0X3RFHTVE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:24:42.110" grpc.time_ms=18.488 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.387851] gitaly[892]: time="2026-09-23T19:24:42.137Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:25:12.035" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:42.135" grpc.time_ms=0.983 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.401596] gitaly[892]: time="2026-09-23T19:24:42.181Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMBCDDC8T29VMS4JM9DCA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:12.079" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:42.180" grpc.time_ms=0.887 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.413994] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.196Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7d8593c892d4f49599958ffd","created_at":"2026-09-23T19:24:40.429Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-303ce4d9d071455c","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:40.441Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3453,"job_status":"rescheduled","message":"PipelineProcessWorker JID-7d8593c892d4f49599958ffd: rescheduled"} gitlab # [ 431.428765] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.205Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7d8593c892d4f49599958ffd","created_at":"2026-09-23T19:24:40.429Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-303ce4d9d071455c","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:40.441Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7d8593c892d4f49599958ffd: done: 0.386613 sec","job_status":"done","queue_duration_s":1.377161,"scheduling_latency_s":1.377192,"gitaly_calls":3,"gitaly_duration_s":0.122546,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.016635,"redis_read_bytes":148,"redis_write_bytes":18013,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.00097,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001093,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00393,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.006768,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000479,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000455,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.016,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.016,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.204222,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.32823796,"duration_s":0.386613,"completed_at":"2026-09-23T19:24:42.205Z","load_balancing_strategy":"primary","db_duration_s":0.056079,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 431.480415] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.207Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","85fd327b0f51aa2aa28a89ce"],"class":"WebHooks::LogExecutionWorker","jid":"1245ccfe5ed489b86ca1b4fd","created_at":"2026-09-23T19:24:41.604Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-f736ca8ff333424e","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:71d0c8625acc0614ed2633d08eaa7eaa75afa0d0afc04bd6403cf2ae69943b11","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:41.607Z","job_size_bytes":2592,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-1245ccfe5ed489b86ca1b4fd: start","job_status":"start","queue_duration_s":0.599643,"scheduling_latency_s":0.599675} gitlab # [ 431.496853] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.243Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","85fd327b0f51aa2aa28a89ce"],"class":"WebHooks::LogExecutionWorker","jid":"1245ccfe5ed489b86ca1b4fd","created_at":"2026-09-23T19:24:41.604Z","trace_propagation_headers":{"sentry-trace":"0428758762f940a78f90529897ce5d65-f736ca8ff333424e","baggage":"sentry-trace_id=0428758762f940a78f90529897ce5d65,sentry-release=Unknown"},"correlation_id":"01M37VM41Y0KDYE5SH42XYSC07","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:71d0c8625acc0614ed2633d08eaa7eaa75afa0d0afc04bd6403cf2ae69943b11","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:41.607Z","job_size_bytes":2592,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-1245ccfe5ed489b86ca1b4fd: done: 0.036352 sec","job_status":"done","queue_duration_s":0.599643,"scheduling_latency_s":0.599675,"redis_calls":7,"redis_duration_s":0.001456,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036352,"completed_at":"2026-09-23T19:24:42.243Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.539329] gitaly[892]: time="2026-09-23T19:24:42.251Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:12.150" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:42.250" grpc.time_ms=0.791 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635' gitlab # [ 431.551997] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.245Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"36684f7b197a847c3538a3a9","created_at":"2026-09-23T19:24:41.710Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-5af484c3758b4492","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:41.721Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-36684f7b197a847c3538a3a9: start","job_status":"start","queue_duration_s":0.524752,"scheduling_latency_s":0.524779} gitlab # [ 431.568268] gitaly[892]: time="2026-09-23T19:24:42.288Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-23T19:25:12.165" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:42.265" grpc.time_ms=23.104 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.580865] gitaly[892]: time="2026-09-23T19:24:42.312Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:24:52.245" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:42.295" grpc.time_ms=16.765 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635', in 0.07 seconds) (finished: subtest: push delivers the webhook, in 87.96 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 431.596690] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.369Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"36684f7b197a847c3538a3a9","created_at":"2026-09-23T19:24:41.710Z","trace_propagation_headers":{"sentry-trace":"cadba7a0b2ab4dd1ba1815638c7848b2-5af484c3758b4492","baggage":"sentry-trace_id=cadba7a0b2ab4dd1ba1815638c7848b2,sentry-release=Unknown"},"correlation_id":"01M37VHJX37KHN0WW0ANV80CSP","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:41.721Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-36684f7b197a847c3538a3a9: done: 0.123942 sec","job_status":"done","queue_duration_s":0.524752,"scheduling_latency_s":0.524779,"redis_calls":11,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000498,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00216,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000367,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003325,"duration_s":0.123942,"completed_at":"2026-09-23T19:24:42.369Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.642797] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.371Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"64dac26271a7bd0897df0582","created_at":"2026-09-23T19:24:41.928Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:41.933Z","job_size_bytes":23,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-64dac26271a7bd0897df0582: start","job_status":"start","queue_duration_s":0.437881,"scheduling_latency_s":0.437909} gitlab # [ 431.659889] gitaly[892]: time="2026-09-23T19:24:42.335Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-23T19:24:52.284" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:42.335" grpc.time_ms=0.269 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.671573] nixbot[3587]: INFO: ::ffff:127.0.0.1:59850 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 200 OK gitlab # [ 431.673632] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.450Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"64dac26271a7bd0897df0582","created_at":"2026-09-23T19:24:41.928Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:41.933Z","job_size_bytes":23,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-64dac26271a7bd0897df0582: done: 0.078806 sec","job_status":"done","queue_duration_s":0.437881,"scheduling_latency_s":0.437909,"redis_calls":7,"redis_duration_s":0.001984,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000494,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.078806,"completed_at":"2026-09-23T19:24:42.450Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011774,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 431.716036] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.456Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f6dd5257d872af79cf2cde92","created_at":"2026-09-23T19:24:42.063Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.066Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f6dd5257d872af79cf2cde92: start","job_status":"start","queue_duration_s":0.390005,"scheduling_latency_s":0.39004} gitlab # [ 431.732644] gitaly[892]: time="2026-09-23T19:24:42.359Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-23T19:24:52.294" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:42.344" grpc.time_ms=14.775 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.746000] gitaly[892]: time="2026-09-23T19:24:42.408Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4151 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:42.407" grpc.time_ms=0.514 pid=892 span.kind=server system=grpc gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.14 seconds) gitlab # [ 431.755563] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.491Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f6dd5257d872af79cf2cde92","created_at":"2026-09-23T19:24:42.063Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.066Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f6dd5257d872af79cf2cde92: done: 0.035166 sec","job_status":"done","queue_duration_s":0.390005,"scheduling_latency_s":0.39004,"redis_calls":7,"redis_duration_s":0.001313,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000347,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035166,"completed_at":"2026-09-23T19:24:42.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.798541] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.493Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c534cde4e800ef2ea1fbcc24","created_at":"2026-09-23T19:24:42.076Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.080Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c534cde4e800ef2ea1fbcc24: start","job_status":"start","queue_duration_s":0.413486,"scheduling_latency_s":0.413514} gitlab # [ 431.814715] gitaly[892]: time="2026-09-23T19:24:42.442Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:42.441" grpc.time_ms=0.536 pid=892 span.kind=server system=grpc gitlab # [ 431.824451] gitaly[892]: time="2026-09-23T19:24:42.449Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX diskcache=4e5041ff-4cee-4e12-a845-a568c205f7b6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:24:52.274" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:42.364" pid=892 ref=refs/keep-around/e6cb306e04578deaef228fc2a54c36a018a70635 remote_ip="2001:db8:1::1" revision=e6cb306e04578deaef228fc2a54c36a018a70635 span.kind=server system=grpc username=root gitlab # [ 431.835833] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 95 gitlab # [ 431.838575] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:42 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 402 gitlab # [ 431.841144] nixbot[3587]: {"ts": "2026-09-23T19:24:42.583Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 \"HTTP/1.1 201 Created\""} gitlab # [ 431.843848] nixbot[3587]: {"ts": "2026-09-23T19:24:42.592Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-e6cb306e0457-5"} gitlab # [ 431.847372] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c534cde4e800ef2ea1fbcc24","created_at":"2026-09-23T19:24:42.076Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.080Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c534cde4e800ef2ea1fbcc24: done: 0.042486 sec","job_status":"done","queue_duration_s":0.413486,"scheduling_latency_s":0.413514,"redis_calls":7,"redis_duration_s":0.001415,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042486,"completed_at":"2026-09-23T19:24:42.536Z","load_balancing_strategy":"primary","db_duration_s":0.008188,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.888965] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.537Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"14009d84960ad7179ddd4524","created_at":"2026-09-23T19:24:42.090Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.093Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-14009d84960ad7179ddd4524: start","job_status":"start","queue_duration_s":0.444016,"scheduling_latency_s":0.444045} gitlab # [ 431.904555] gitaly[892]: time="2026-09-23T19:24:42.449Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5013 command.oublock=8 command.real_time_ms=78 command.spawn_token_fork_ms=1 command.system_time_ms=34 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:24:52.274" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:24:42.364" grpc.time_ms=84.797 pid=892 ref=refs/keep-around/e6cb306e04578deaef228fc2a54c36a018a70635 remote_ip="2001:db8:1::1" revision=e6cb306e04578deaef228fc2a54c36a018a70635 span.kind=server system=grpc username=root gitlab # [ 431.919080] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.576Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"14009d84960ad7179ddd4524","created_at":"2026-09-23T19:24:42.090Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.093Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-14009d84960ad7179ddd4524: done: 0.039296 sec","job_status":"done","queue_duration_s":0.444016,"scheduling_latency_s":0.444045,"redis_calls":10,"redis_duration_s":0.0017699999999999999,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000605,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000341,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039296,"completed_at":"2026-09-23T19:24:42.576Z","load_balancing_strategy":"primary","db_duration_s":0.008089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.961758] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.578Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6ee4c9654a89aa94c6f64c9e","created_at":"2026-09-23T19:24:42.094Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.096Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6ee4c9654a89aa94c6f64c9e: start","job_status":"start","queue_duration_s":0.481213,"scheduling_latency_s":0.48124} gitlab # [ 431.978201] gitaly[892]: time="2026-09-23T19:24:42.456Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:12.355" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:42.456" grpc.time_ms=0.359 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.990788] nixbot[3587]: {"ts": "2026-09-23T19:24:42.752Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-e6cb306e0457-5", "argv0": "bwrap"} gitlab # [ 431.993186] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.608Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6ee4c9654a89aa94c6f64c9e","created_at":"2026-09-23T19:24:42.094Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.096Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6ee4c9654a89aa94c6f64c9e: done: 0.030751 sec","job_status":"done","queue_duration_s":0.481213,"scheduling_latency_s":0.48124,"redis_calls":7,"redis_duration_s":0.0013610000000000002,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000362,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030751,"completed_at":"2026-09-23T19:24:42.608Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.035338] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.610Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"81bbdbd406692fde1608de44","created_at":"2026-09-23T19:24:42.098Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.101Z","job_size_bytes":19,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-81bbdbd406692fde1608de44: start","job_status":"start","queue_duration_s":0.508973,"scheduling_latency_s":0.509004} gitlab # [ 432.052613] gitaly[892]: time="2026-09-23T19:24:42.523Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMDJQA2KMRRWWD2HED8D7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:12.522" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:42.522" grpc.time_ms=0.653 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.065346] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.640Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"81bbdbd406692fde1608de44","created_at":"2026-09-23T19:24:42.098Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.101Z","job_size_bytes":19,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-81bbdbd406692fde1608de44: done: 0.029447 sec","job_status":"done","queue_duration_s":0.508973,"scheduling_latency_s":0.509004,"redis_calls":8,"redis_duration_s":0.0017560000000000002,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029447,"completed_at":"2026-09-23T19:24:42.640Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.107997] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.641Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d1349afc92c3b965da8485b2","created_at":"2026-09-23T19:24:42.103Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.106Z","job_size_bytes":66,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d1349afc92c3b965da8485b2: start","job_status":"start","queue_duration_s":0.535445,"scheduling_latency_s":0.535465} gitlab # [ 432.124933] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.668Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d1349afc92c3b965da8485b2","created_at":"2026-09-23T19:24:42.103Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.106Z","job_size_bytes":66,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d1349afc92c3b965da8485b2: done: 0.027141 sec","job_status":"done","queue_duration_s":0.535445,"scheduling_latency_s":0.535465,"redis_calls":7,"redis_duration_s":0.001402,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00036,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027141,"completed_at":"2026-09-23T19:24:42.668Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.169344] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.670Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7a3c41e693036e9df61490e0","created_at":"2026-09-23T19:24:42.107Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.109Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7a3c41e693036e9df61490e0: start","job_status":"start","queue_duration_s":0.560758,"scheduling_latency_s":0.560777} gitlab # [ 432.186169] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.701Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7a3c41e693036e9df61490e0","created_at":"2026-09-23T19:24:42.107Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.109Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7a3c41e693036e9df61490e0: done: 0.031152 sec","job_status":"done","queue_duration_s":0.560758,"scheduling_latency_s":0.560777,"redis_calls":7,"redis_duration_s":0.001356,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031152,"completed_at":"2026-09-23T19:24:42.701Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.228728] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.702Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3c12d7e592680ef1b266b734","created_at":"2026-09-23T19:24:42.110Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.113Z","job_size_bytes":6,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3c12d7e592680ef1b266b734: start","job_status":"start","queue_duration_s":0.589054,"scheduling_latency_s":0.589073} gitlab # [ 432.244663] nix-daemon[3757]: accepted connection from pid 4796, user nixbot gitlab # [ 432.245639] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3c12d7e592680ef1b266b734","created_at":"2026-09-23T19:24:42.110Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.113Z","job_size_bytes":6,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3c12d7e592680ef1b266b734: done: 0.015231 sec","job_status":"done","queue_duration_s":0.589054,"scheduling_latency_s":0.589073,"redis_calls":7,"redis_duration_s":0.0014000000000000002,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015231,"completed_at":"2026-09-23T19:24:42.717Z","load_balancing_strategy":"primary","db_duration_s":0.00033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.289242] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.719Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"79e7b844bd10424fdf1ef33a","created_at":"2026-09-23T19:24:42.114Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.117Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-79e7b844bd10424fdf1ef33a: start","job_status":"start","queue_duration_s":0.601344,"scheduling_latency_s":0.601363} gitlab # [ 432.304522] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.744Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"79e7b844bd10424fdf1ef33a","created_at":"2026-09-23T19:24:42.114Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.117Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-79e7b844bd10424fdf1ef33a: done: 0.025287 sec","job_status":"done","queue_duration_s":0.601344,"scheduling_latency_s":0.601363,"redis_calls":7,"redis_duration_s":0.001603,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025287,"completed_at":"2026-09-23T19:24:42.744Z","load_balancing_strategy":"primary","db_duration_s":0.00305,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.346016] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.749Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"5fe82ef8879d1266de9ddaa8","created_at":"2026-09-23T19:24:42.197Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.199Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5fe82ef8879d1266de9ddaa8: start","job_status":"start","queue_duration_s":0.550715,"scheduling_latency_s":0.550736} gitlab # [ 432.361889] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.780Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"5fe82ef8879d1266de9ddaa8","created_at":"2026-09-23T19:24:42.197Z","trace_propagation_headers":{"sentry-trace":"f260efc97d314fcfacba4cf67affe178-cbb10343ca644297","baggage":"sentry-trace_id=f260efc97d314fcfacba4cf67affe178,sentry-release=Unknown"},"correlation_id":"01M37VMBCDDC8T29VMS4JM9DCA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.199Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5fe82ef8879d1266de9ddaa8: done: 0.030464 sec","job_status":"done","queue_duration_s":0.550715,"scheduling_latency_s":0.550736,"redis_calls":8,"redis_duration_s":0.0020340000000000002,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030464,"completed_at":"2026-09-23T19:24:42.780Z","load_balancing_strategy":"primary","db_duration_s":0.006574,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 432.403671] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.781Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8245e80054830f19f5f23072","created_at":"2026-09-23T19:24:42.470Z","trace_propagation_headers":{"sentry-trace":"be4513a1256347d484dc44752a51be14-bb49f06d90fe424c","baggage":"sentry-trace_id=be4513a1256347d484dc44752a51be14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMDA2TKBCXX8EKAMF7WWX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.476Z","job_size_bytes":65,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8245e80054830f19f5f23072: start","job_status":"start","queue_duration_s":0.305229,"scheduling_latency_s":0.305248} gitlab # [ 432.420564] gitaly[892]: time="2026-09-23T19:24:43.119Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:25:13.017" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:43.118" grpc.time_ms=1.683 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.433039] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.818Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8245e80054830f19f5f23072","created_at":"2026-09-23T19:24:42.470Z","trace_propagation_headers":{"sentry-trace":"be4513a1256347d484dc44752a51be14-bb49f06d90fe424c","baggage":"sentry-trace_id=be4513a1256347d484dc44752a51be14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMDA2TKBCXX8EKAMF7WWX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.476Z","job_size_bytes":65,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8245e80054830f19f5f23072: done: 0.036591 sec","job_status":"done","queue_duration_s":0.305229,"scheduling_latency_s":0.305248,"redis_calls":7,"redis_duration_s":0.002297,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036591,"completed_at":"2026-09-23T19:24:42.818Z","load_balancing_strategy":"primary","db_duration_s":0.010227,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 432.475530] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.819Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"964bf8a99efa990598dcbd7f","created_at":"2026-09-23T19:24:42.477Z","trace_propagation_headers":{"sentry-trace":"be4513a1256347d484dc44752a51be14-bb49f06d90fe424c","baggage":"sentry-trace_id=be4513a1256347d484dc44752a51be14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMDA2TKBCXX8EKAMF7WWX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.482Z","job_size_bytes":65,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-964bf8a99efa990598dcbd7f: start","job_status":"start","queue_duration_s":0.337656,"scheduling_latency_s":0.337675} gitlab # [ 432.493988] gitaly[892]: time="2026-09-23T19:24:43.187Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMDA2TKBCXX8EKAMF7WWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:13.084" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:43.185" grpc.time_ms=1.669 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.506326] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.956Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"964bf8a99efa990598dcbd7f","created_at":"2026-09-23T19:24:42.477Z","trace_propagation_headers":{"sentry-trace":"be4513a1256347d484dc44752a51be14-bb49f06d90fe424c","baggage":"sentry-trace_id=be4513a1256347d484dc44752a51be14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMDA2TKBCXX8EKAMF7WWX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.482Z","job_size_bytes":65,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-964bf8a99efa990598dcbd7f: done: 0.137006 sec","job_status":"done","queue_duration_s":0.337656,"scheduling_latency_s":0.337675,"redis_calls":10,"redis_duration_s":0.002937,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001607,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00133,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.137006,"completed_at":"2026-09-23T19:24:42.956Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.549569] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:42.963Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"25524e549e07bb1a562035ae","created_at":"2026-09-23T19:24:42.559Z","trace_propagation_headers":{"sentry-trace":"be4513a1256347d484dc44752a51be14-bb49f06d90fe424c","baggage":"sentry-trace_id=be4513a1256347d484dc44752a51be14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMDA2TKBCXX8EKAMF7WWX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.563Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-25524e549e07bb1a562035ae: start","job_status":"start","queue_duration_s":0.399301,"scheduling_latency_s":0.399349} gitlab # [ 432.565262] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:43.215Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"25524e549e07bb1a562035ae","created_at":"2026-09-23T19:24:42.559Z","trace_propagation_headers":{"sentry-trace":"be4513a1256347d484dc44752a51be14-bb49f06d90fe424c","baggage":"sentry-trace_id=be4513a1256347d484dc44752a51be14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMDA2TKBCXX8EKAMF7WWX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:42.563Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-25524e549e07bb1a562035ae: done: 0.252926 sec","job_status":"done","queue_duration_s":0.399301,"scheduling_latency_s":0.399349,"gitaly_calls":2,"gitaly_duration_s":0.011515,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.016044,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001997,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002005,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000433,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.006223,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000445,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000962,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001982,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.187543,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.17259918,"duration_s":0.252926,"completed_at":"2026-09-23T19:24:43.215Z","load_balancing_strategy":"primary","db_duration_s":0.047529,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 432.615521] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:43.219Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a594541b9b8a87aa073f04a8","created_at":"2026-09-23T19:24:43.089Z","trace_propagation_headers":{"sentry-trace":"be4513a1256347d484dc44752a51be14-1370d69fad174d35","baggage":"sentry-trace_id=be4513a1256347d484dc44752a51be14,sentry-release=Unknown"},"correlation_id":"01M37VMDA2TKBCXX8EKAMF7WWX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:43.092Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a594541b9b8a87aa073f04a8: start","job_status":"start","queue_duration_s":0.126511,"scheduling_latency_s":0.126563} gitlab # [ 432.638239] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:43.433Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a594541b9b8a87aa073f04a8","created_at":"2026-09-23T19:24:43.089Z","trace_propagation_headers":{"sentry-trace":"be4513a1256347d484dc44752a51be14-1370d69fad174d35","baggage":"sentry-trace_id=be4513a1256347d484dc44752a51be14,sentry-release=Unknown"},"correlation_id":"01M37VMDA2TKBCXX8EKAMF7WWX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:43.092Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a594541b9b8a87aa073f04a8: done: 0.214535 sec","job_status":"done","queue_duration_s":0.126511,"scheduling_latency_s":0.126563,"redis_calls":7,"redis_duration_s":0.003219,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.195184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.214535,"completed_at":"2026-09-23T19:24:43.433Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 432.968227] gitaly[892]: time="2026-09-23T19:24:43.763Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMEQZ66H1SGPSKY1K5QEB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:13.661" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:43.762" grpc.time_ms=1.32 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.27 seconds) gitlab # [ 433.030245] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 178 gitlab # [ 433.701204] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:44.496Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"af6c3f5234155b4370745b56","created_at":"2026-09-23T19:24:44.463Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.490Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-af6c3f5234155b4370745b56: start","job_status":"start","queue_duration_s":0.005406,"scheduling_latency_s":0.00547} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 434.182992] gitaly[892]: time="2026-09-23T19:24:44.978Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMFYK82YS8C9TCTVEBRSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:14.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:44.976" grpc.time_ms=1.324 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.21 seconds) gitlab # [ 434.236743] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 148 gitlab # [ 434.479559] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:45.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"af6c3f5234155b4370745b56","created_at":"2026-09-23T19:24:44.463Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.490Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-af6c3f5234155b4370745b56: done: 0.779229 sec","job_status":"done","queue_duration_s":0.005406,"scheduling_latency_s":0.00547,"redis_calls":70,"redis_duration_s":0.326031,"redis_read_bytes":118,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.025521,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.003098,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.289436,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.007976,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.321739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.779229,"completed_at":"2026-09-23T19:24:45.274Z","load_balancing_strategy":"primary","db_duration_s":0.020371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.584678] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:45.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6070f40be951b47bc8aed6f8","created_at":"2026-09-23T19:24:44.511Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a3b86828f0edd7b0022fc2a4d10c540","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.538Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6070f40be951b47bc8aed6f8: start","job_status":"start","queue_duration_s":0.742899,"scheduling_latency_s":0.742962} gitlab # [ 434.633493] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:45.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6070f40be951b47bc8aed6f8","created_at":"2026-09-23T19:24:44.511Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a3b86828f0edd7b0022fc2a4d10c540","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.538Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6070f40be951b47bc8aed6f8: done: 0.147648 sec","job_status":"done","queue_duration_s":0.742899,"scheduling_latency_s":0.742962,"redis_calls":12,"redis_duration_s":0.038976,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002531,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.025719,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.010726,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.02660678,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.147648,"completed_at":"2026-09-23T19:24:45.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010424,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 434.694388] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:45.439Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c366535184e20f331e32b2b5","created_at":"2026-09-23T19:24:44.559Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7cf026d991ac6b8a0bcdfec40e4666a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.585Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c366535184e20f331e32b2b5: start","job_status":"start","queue_duration_s":0.854319,"scheduling_latency_s":0.85438} gitlab # [ 435.023266] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:45.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c366535184e20f331e32b2b5","created_at":"2026-09-23T19:24:44.559Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7cf026d991ac6b8a0bcdfec40e4666a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.585Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c366535184e20f331e32b2b5: done: 0.377184 sec","job_status":"done","queue_duration_s":0.854319,"scheduling_latency_s":0.85438,"redis_calls":13,"redis_duration_s":0.075556,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.008367,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010027,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.057162,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00396424,"exclusive_lock_hold_duration_s":0.13519456,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.377184,"completed_at":"2026-09-23T19:24:45.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.073739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.111919] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:45.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"59365abb70ea3865c1a114dd","created_at":"2026-09-23T19:24:44.605Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40816eeba6b1d9b3dd97beb400c85b40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.632Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-59365abb70ea3865c1a114dd: start","job_status":"start","queue_duration_s":1.198217,"scheduling_latency_s":1.198268} gitlab # [ 435.160421] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:45.957Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 435.205076] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"59365abb70ea3865c1a114dd","created_at":"2026-09-23T19:24:44.605Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40816eeba6b1d9b3dd97beb400c85b40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.632Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-59365abb70ea3865c1a114dd: done: 0.170399 sec","job_status":"done","queue_duration_s":1.198217,"scheduling_latency_s":1.198268,"redis_calls":10,"redis_duration_s":0.032216999999999996,"redis_read_bytes":41,"redis_write_bytes":1140,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.017913,"redis_feature_flag_read_bytes":36,"redis_feature_flag_write_bytes":374,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007894,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.170399,"completed_at":"2026-09-23T19:24:46.000Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.308778] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e32d0a8e93a27064cd65f107","created_at":"2026-09-23T19:24:44.671Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ed9d8cc229c80e606452820eeecddd6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.696Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e32d0a8e93a27064cd65f107: start","job_status":"start","queue_duration_s":1.31416,"scheduling_latency_s":1.31422} gitlab # [ 435.329362] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.106Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 435.334130] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.127Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e32d0a8e93a27064cd65f107","created_at":"2026-09-23T19:24:44.671Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ed9d8cc229c80e606452820eeecddd6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.696Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e32d0a8e93a27064cd65f107: done: 0.117014 sec","job_status":"done","queue_duration_s":1.31416,"scheduling_latency_s":1.31422,"redis_calls":9,"redis_duration_s":0.01291,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003977,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007818,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.117014,"completed_at":"2026-09-23T19:24:46.127Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.380731] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.130Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"457f3301fd948d36afb16001","created_at":"2026-09-23T19:24:44.726Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49da62009b470c28a39363228c3dc03f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:24:44.753Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-457f3301fd948d36afb16001: start","job_status":"start","queue_duration_s":1.377813,"scheduling_latency_s":1.377875} gitlab # [ 435.394025] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.161Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.19 seconds) gitlab # [ 435.397317] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"457f3301fd948d36afb16001","created_at":"2026-09-23T19:24:44.726Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49da62009b470c28a39363228c3dc03f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:24:44.753Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-457f3301fd948d36afb16001: done: 0.052783 sec","job_status":"done","queue_duration_s":1.377813,"scheduling_latency_s":1.377875,"redis_calls":9,"redis_duration_s":0.0040680000000000004,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000917,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002226,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052783,"completed_at":"2026-09-23T19:24:46.183Z","load_balancing_strategy":"primary","db_duration_s":0.001163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.438997] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.187Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2368f91a30d7f09fc2119bdc","created_at":"2026-09-23T19:24:44.804Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64fb7fdaa25bd6ed65d7c061ea63a8ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.832Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2368f91a30d7f09fc2119bdc: start","job_status":"start","queue_duration_s":1.354324,"scheduling_latency_s":1.35438} gitlab # [ 435.454658] gitaly[892]: time="2026-09-23T19:24:46.193Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMH51CWDJHY7KR7TXJ35E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:16.190" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:46.191" grpc.time_ms=1.183 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.468222] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 109 gitlab # [ 435.470903] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2368f91a30d7f09fc2119bdc","created_at":"2026-09-23T19:24:44.804Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64fb7fdaa25bd6ed65d7c061ea63a8ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.832Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2368f91a30d7f09fc2119bdc: done: 0.048346 sec","job_status":"done","queue_duration_s":1.354324,"scheduling_latency_s":1.35438,"redis_calls":11,"redis_duration_s":0.005341,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00167,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048346,"completed_at":"2026-09-23T19:24:46.235Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001766,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.513993] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"ad8b770a653f8ab310f6276b","created_at":"2026-09-23T19:24:44.865Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7a11d0e293f46c4949d0d47776f1a72","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.897Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-ad8b770a653f8ab310f6276b: start","job_status":"start","queue_duration_s":1.340482,"scheduling_latency_s":1.340526} gitlab # [ 435.528902] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.287Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"ad8b770a653f8ab310f6276b","created_at":"2026-09-23T19:24:44.865Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7a11d0e293f46c4949d0d47776f1a72","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.897Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-ad8b770a653f8ab310f6276b: done: 0.048833 sec","job_status":"done","queue_duration_s":1.340482,"scheduling_latency_s":1.340526,"redis_calls":9,"redis_duration_s":0.003484,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000839,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001924,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000721,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048833,"completed_at":"2026-09-23T19:24:46.287Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.572243] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"995d00cdb851bbcac4607d54","created_at":"2026-09-23T19:24:44.920Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"520b9560c53359565255ce040ba8a334","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.947Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-995d00cdb851bbcac4607d54: start","job_status":"start","queue_duration_s":1.342837,"scheduling_latency_s":1.342875} gitlab # [ 435.588334] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"995d00cdb851bbcac4607d54","created_at":"2026-09-23T19:24:44.920Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"520b9560c53359565255ce040ba8a334","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:44.947Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-995d00cdb851bbcac4607d54: done: 0.028268 sec","job_status":"done","queue_duration_s":1.342837,"scheduling_latency_s":1.342875,"redis_calls":9,"redis_duration_s":0.002628,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000654,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001424,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00055,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028268,"completed_at":"2026-09-23T19:24:46.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.631204] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.320Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"6d9439811538320978f30fec","created_at":"2026-09-23T19:24:44.974Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c391d7cec59c77ce1553d81286c3a25","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.004Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-6d9439811538320978f30fec: start","job_status":"start","queue_duration_s":1.315217,"scheduling_latency_s":1.315242} gitlab # [ 435.646019] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"6d9439811538320978f30fec","created_at":"2026-09-23T19:24:44.974Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c391d7cec59c77ce1553d81286c3a25","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.004Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-6d9439811538320978f30fec: done: 0.025195 sec","job_status":"done","queue_duration_s":1.315217,"scheduling_latency_s":1.315242,"redis_calls":12,"redis_duration_s":0.003366,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000624,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001555,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001187,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041498,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.025195,"completed_at":"2026-09-23T19:24:46.345Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.691875] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4cceebbaa28dd165cc622074","created_at":"2026-09-23T19:24:45.027Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75bc458abe46e1c2ca5acec4db814c43","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.049Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4cceebbaa28dd165cc622074: start","job_status":"start","queue_duration_s":1.298322,"scheduling_latency_s":1.298345} gitlab # [ 436.044805] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"4cceebbaa28dd165cc622074","created_at":"2026-09-23T19:24:45.027Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75bc458abe46e1c2ca5acec4db814c43","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.049Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-4cceebbaa28dd165cc622074: done: 0.492592 sec","job_status":"done","queue_duration_s":1.298322,"scheduling_latency_s":1.298345,"redis_calls":564,"redis_duration_s":0.126004,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001521,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.123871,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.418381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.492592,"completed_at":"2026-09-23T19:24:46.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.140636] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:46.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a8ab02ca6116ebacd0991c62","created_at":"2026-09-23T19:24:45.073Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9e8424416433f1bbcceb404055cb959","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.102Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a8ab02ca6116ebacd0991c62: start","job_status":"start","queue_duration_s":1.739682,"scheduling_latency_s":1.739714} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 436.331993] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.128Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a8ab02ca6116ebacd0991c62","created_at":"2026-09-23T19:24:45.073Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9e8424416433f1bbcceb404055cb959","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.102Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a8ab02ca6116ebacd0991c62: done: 0.28649 sec","job_status":"done","queue_duration_s":1.739682,"scheduling_latency_s":1.739714,"redis_calls":13,"redis_duration_s":0.0036839999999999998,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001016,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001357,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.055,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.055,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.213234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070044,"exclusive_lock_hold_duration_s":0.2574658,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.28649,"completed_at":"2026-09-23T19:24:47.128Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.081563,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.444512] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.130Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Environments::AutoStopCronWorker","jid":"34dcd8c0285b9ccf55c6a6be","created_at":"2026-09-23T19:24:45.146Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f250a2e96a498ff6604afb280c32efe","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_delivery","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:037d6a92006e3beb0e73d758a3f9a2a2a104eaa8b9d3c2b1a2cf37915906a513","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.176Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Environments::AutoStopCronWorker JID-34dcd8c0285b9ccf55c6a6be: start","job_status":"start","queue_duration_s":1.953662,"scheduling_latency_s":1.953685} gitlab # [ 436.526196] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.322Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Environments::AutoStopCronWorker","jid":"34dcd8c0285b9ccf55c6a6be","created_at":"2026-09-23T19:24:45.146Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f250a2e96a498ff6604afb280c32efe","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_delivery","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:037d6a92006e3beb0e73d758a3f9a2a2a104eaa8b9d3c2b1a2cf37915906a513","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.176Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Environments::AutoStopCronWorker JID-34dcd8c0285b9ccf55c6a6be: done: 0.192477 sec","job_status":"done","queue_duration_s":1.953662,"scheduling_latency_s":1.953685,"redis_calls":12,"redis_duration_s":0.003248,"redis_read_bytes":11,"redis_write_bytes":1368,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000574,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001208,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":503,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.172978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00071672,"exclusive_lock_hold_duration_s":0.01265982,"duration_s":0.192477,"completed_at":"2026-09-23T19:24:47.322Z","load_balancing_strategy":"primary","db_duration_s":0.007487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.599584] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df744765375377b8204fbeec","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df744765375377b8204fbeec: start","job_status":"start","queue_duration_s":2.087849,"scheduling_latency_s":2.087878} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.20 seconds) gitlab # [ 436.615498] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df744765375377b8204fbeec","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df744765375377b8204fbeec: done: 0.035312 sec","job_status":"done","queue_duration_s":2.087849,"scheduling_latency_s":2.087878,"redis_calls":15,"redis_duration_s":0.004698,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0007,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035312,"completed_at":"2026-09-23T19:24:47.360Z","load_balancing_strategy":"primary","db_duration_s":0.001011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.664301] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3890fe9f78c82274e2bee66b","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3890fe9f78c82274e2bee66b: start","job_status":"start","queue_duration_s":2.126643,"scheduling_latency_s":2.12668} gitlab # [ 436.677706] gitaly[892]: time="2026-09-23T19:24:47.382Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMJA1ZD4B5KB7R0P2ZKES grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:17.279" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:47.381" grpc.time_ms=1.175 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.691145] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 124 gitlab # [ 436.693883] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3890fe9f78c82274e2bee66b","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3890fe9f78c82274e2bee66b: done: 0.026835 sec","job_status":"done","queue_duration_s":2.126643,"scheduling_latency_s":2.12668,"redis_calls":13,"redis_duration_s":0.003664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002471,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026835,"completed_at":"2026-09-23T19:24:47.390Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.733209] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d15a75ce7690e71d9e5be20","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d15a75ce7690e71d9e5be20: start","job_status":"start","queue_duration_s":2.155295,"scheduling_latency_s":2.155319} gitlab # [ 436.747119] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d15a75ce7690e71d9e5be20","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d15a75ce7690e71d9e5be20: done: 0.022567 sec","job_status":"done","queue_duration_s":2.155295,"scheduling_latency_s":2.155319,"redis_calls":13,"redis_duration_s":0.002515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001529,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022567,"completed_at":"2026-09-23T19:24:47.415Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.787316] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24279b7019f56b8d61c52bd2","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24279b7019f56b8d61c52bd2: start","job_status":"start","queue_duration_s":2.179985,"scheduling_latency_s":2.180014} gitlab # [ 436.801404] nix-daemon[3757]: accepted connection from pid 4834, user nixbot gitlab # [ 436.802352] nix-daemon[3757]: accepted connection from pid 4796, user nixbot gitlab # [ 436.803315] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24279b7019f56b8d61c52bd2","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24279b7019f56b8d61c52bd2: done: 0.024609 sec","job_status":"done","queue_duration_s":2.179985,"scheduling_latency_s":2.180014,"redis_calls":13,"redis_duration_s":0.002315,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001419,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024609,"completed_at":"2026-09-23T19:24:47.441Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.842513] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"707af881e6353b8f87b36d82","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-707af881e6353b8f87b36d82: start","job_status":"start","queue_duration_s":2.206881,"scheduling_latency_s":2.206916} gitlab # [ 436.855077] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"707af881e6353b8f87b36d82","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-707af881e6353b8f87b36d82: done: 0.024746 sec","job_status":"done","queue_duration_s":2.206881,"scheduling_latency_s":2.206916,"redis_calls":13,"redis_duration_s":0.0023940000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001577,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024746,"completed_at":"2026-09-23T19:24:47.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.894475] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"918a50ce5637ddce74c39039","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-918a50ce5637ddce74c39039: start","job_status":"start","queue_duration_s":2.233103,"scheduling_latency_s":2.233133} gitlab # [ 436.906987] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"918a50ce5637ddce74c39039","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-918a50ce5637ddce74c39039: done: 0.018009 sec","job_status":"done","queue_duration_s":2.233103,"scheduling_latency_s":2.233133,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018009,"completed_at":"2026-09-23T19:24:47.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.946120] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7e7bd9c054aa7d15f847bee","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7e7bd9c054aa7d15f847bee: start","job_status":"start","queue_duration_s":2.2525,"scheduling_latency_s":2.252528} gitlab # [ 436.958578] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7e7bd9c054aa7d15f847bee","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7e7bd9c054aa7d15f847bee: done: 0.01742 sec","job_status":"done","queue_duration_s":2.2525,"scheduling_latency_s":2.252528,"redis_calls":13,"redis_duration_s":0.0015819999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01742,"completed_at":"2026-09-23T19:24:47.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.997622] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffa66fa21eebab5e1341fe9c","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffa66fa21eebab5e1341fe9c: start","job_status":"start","queue_duration_s":2.271219,"scheduling_latency_s":2.271246} gitlab # [ 437.011120] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffa66fa21eebab5e1341fe9c","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffa66fa21eebab5e1341fe9c: done: 0.018403 sec","job_status":"done","queue_duration_s":2.271219,"scheduling_latency_s":2.271246,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018403,"completed_at":"2026-09-23T19:24:47.526Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.052111] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff150a43c7a6928024a18461","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff150a43c7a6928024a18461: start","job_status":"start","queue_duration_s":2.29096,"scheduling_latency_s":2.29099} gitlab # [ 437.064344] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff150a43c7a6928024a18461","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff150a43c7a6928024a18461: done: 0.018559 sec","job_status":"done","queue_duration_s":2.29096,"scheduling_latency_s":2.29099,"redis_calls":13,"redis_duration_s":0.001612,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018559,"completed_at":"2026-09-23T19:24:47.546Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.103015] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"852e513e5bf8a0895691f310","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-852e513e5bf8a0895691f310: start","job_status":"start","queue_duration_s":2.310858,"scheduling_latency_s":2.310888} gitlab # [ 437.115251] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"852e513e5bf8a0895691f310","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-852e513e5bf8a0895691f310: done: 0.018166 sec","job_status":"done","queue_duration_s":2.310858,"scheduling_latency_s":2.310888,"redis_calls":13,"redis_duration_s":0.001639,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018166,"completed_at":"2026-09-23T19:24:47.566Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.154033] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86e4724e3336b2f2c113d22f","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86e4724e3336b2f2c113d22f: start","job_status":"start","queue_duration_s":2.332052,"scheduling_latency_s":2.332085} gitlab # [ 437.166238] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86e4724e3336b2f2c113d22f","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86e4724e3336b2f2c113d22f: done: 0.018753 sec","job_status":"done","queue_duration_s":2.332052,"scheduling_latency_s":2.332085,"redis_calls":13,"redis_duration_s":0.001814,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018753,"completed_at":"2026-09-23T19:24:47.588Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.206014] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"290722b0fe8fc7d5afba20d7","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-290722b0fe8fc7d5afba20d7: start","job_status":"start","queue_duration_s":2.352445,"scheduling_latency_s":2.352475} gitlab # [ 437.218962] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"290722b0fe8fc7d5afba20d7","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-290722b0fe8fc7d5afba20d7: done: 0.018593 sec","job_status":"done","queue_duration_s":2.352445,"scheduling_latency_s":2.352475,"redis_calls":13,"redis_duration_s":0.0030299999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018593,"completed_at":"2026-09-23T19:24:47.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.258387] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9e5bfbf82e6abb0983a742a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9e5bfbf82e6abb0983a742a: start","job_status":"start","queue_duration_s":2.37258,"scheduling_latency_s":2.372607} gitlab # [ 437.270980] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9e5bfbf82e6abb0983a742a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9e5bfbf82e6abb0983a742a: done: 0.016865 sec","job_status":"done","queue_duration_s":2.37258,"scheduling_latency_s":2.372607,"redis_calls":13,"redis_duration_s":0.0025960000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00166,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016865,"completed_at":"2026-09-23T19:24:47.626Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.310314] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bd83058f94e16bfc5e27270","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bd83058f94e16bfc5e27270: start","job_status":"start","queue_duration_s":2.392177,"scheduling_latency_s":2.392202} gitlab # [ 437.322932] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bd83058f94e16bfc5e27270","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bd83058f94e16bfc5e27270: done: 0.017323 sec","job_status":"done","queue_duration_s":2.392177,"scheduling_latency_s":2.392202,"redis_calls":13,"redis_duration_s":0.002856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017323,"completed_at":"2026-09-23T19:24:47.646Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.362043] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a5d2916ef45adaf74b89e94","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a5d2916ef45adaf74b89e94: start","job_status":"start","queue_duration_s":2.411051,"scheduling_latency_s":2.411075} gitlab # [ 437.374613] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a5d2916ef45adaf74b89e94","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a5d2916ef45adaf74b89e94: done: 0.017424 sec","job_status":"done","queue_duration_s":2.411051,"scheduling_latency_s":2.411075,"redis_calls":13,"redis_duration_s":0.002682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001776,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017424,"completed_at":"2026-09-23T19:24:47.665Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.414941] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0687b50268206af9378083a1","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0687b50268206af9378083a1: start","job_status":"start","queue_duration_s":2.429985,"scheduling_latency_s":2.430011} gitlab # [ 437.427162] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0687b50268206af9378083a1","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0687b50268206af9378083a1: done: 0.019164 sec","job_status":"done","queue_duration_s":2.429985,"scheduling_latency_s":2.430011,"redis_calls":13,"redis_duration_s":0.003142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019164,"completed_at":"2026-09-23T19:24:47.686Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.465248] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e51cbd34fd3a5a8328653735","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e51cbd34fd3a5a8328653735: start","job_status":"start","queue_duration_s":2.450939,"scheduling_latency_s":2.450967} gitlab # [ 437.477412] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e51cbd34fd3a5a8328653735","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e51cbd34fd3a5a8328653735: done: 0.017485 sec","job_status":"done","queue_duration_s":2.450939,"scheduling_latency_s":2.450967,"redis_calls":13,"redis_duration_s":0.0028859999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017485,"completed_at":"2026-09-23T19:24:47.705Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.515749] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a0cec141c7614b656aa619e","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a0cec141c7614b656aa619e: start","job_status":"start","queue_duration_s":2.469799,"scheduling_latency_s":2.469817} gitlab # [ 437.527951] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a0cec141c7614b656aa619e","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a0cec141c7614b656aa619e: done: 0.016119 sec","job_status":"done","queue_duration_s":2.469799,"scheduling_latency_s":2.469817,"redis_calls":13,"redis_duration_s":0.002742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001784,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016119,"completed_at":"2026-09-23T19:24:47.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.566972] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce99e82f0da4332295abd390","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce99e82f0da4332295abd390: start","job_status":"start","queue_duration_s":2.487369,"scheduling_latency_s":2.487385} gitlab # [ 437.579460] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce99e82f0da4332295abd390","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce99e82f0da4332295abd390: done: 0.016434 sec","job_status":"done","queue_duration_s":2.487369,"scheduling_latency_s":2.487385,"redis_calls":13,"redis_duration_s":0.002935,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016434,"completed_at":"2026-09-23T19:24:47.741Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.619132] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36b1208f2a575b38286b1f84","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36b1208f2a575b38286b1f84: start","job_status":"start","queue_duration_s":2.506308,"scheduling_latency_s":2.506327} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 437.631608] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36b1208f2a575b38286b1f84","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36b1208f2a575b38286b1f84: done: 0.016294 sec","job_status":"done","queue_duration_s":2.506308,"scheduling_latency_s":2.506327,"redis_calls":13,"redis_duration_s":0.002737,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001758,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016294,"completed_at":"2026-09-23T19:24:47.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.667599] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf56fd99f5b3f31503bffee0","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf56fd99f5b3f31503bffee0: start","job_status":"start","queue_duration_s":2.523967,"scheduling_latency_s":2.523984} gitlab # [ 437.679545] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf56fd99f5b3f31503bffee0","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf56fd99f5b3f31503bffee0: done: 0.016255 sec","job_status":"done","queue_duration_s":2.523967,"scheduling_latency_s":2.523984,"redis_calls":13,"redis_duration_s":0.002628,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001714,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016255,"completed_at":"2026-09-23T19:24:47.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.718370] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f675a3c449b7717a2a6cac7a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f675a3c449b7717a2a6cac7a: start","job_status":"start","queue_duration_s":2.543654,"scheduling_latency_s":2.543705} gitlab # [ 437.730723] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f675a3c449b7717a2a6cac7a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f675a3c449b7717a2a6cac7a: done: 0.048155 sec","job_status":"done","queue_duration_s":2.543654,"scheduling_latency_s":2.543705,"redis_calls":13,"redis_duration_s":0.005869,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.048155,"completed_at":"2026-09-23T19:24:47.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.769513] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46970379db9cec14a223fde0","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46970379db9cec14a223fde0: start","job_status":"start","queue_duration_s":2.594854,"scheduling_latency_s":2.594896} gitlab # [ 437.781824] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46970379db9cec14a223fde0","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46970379db9cec14a223fde0: done: 0.033938 sec","job_status":"done","queue_duration_s":2.594854,"scheduling_latency_s":2.594896,"redis_calls":13,"redis_duration_s":0.004005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033938,"completed_at":"2026-09-23T19:24:47.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.820508] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99ee35914f334b41321656ac","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99ee35914f334b41321656ac: start","job_status":"start","queue_duration_s":2.63081,"scheduling_latency_s":2.630836} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.23 seconds) gitlab # [ 437.833824] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99ee35914f334b41321656ac","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99ee35914f334b41321656ac: done: 0.024811 sec","job_status":"done","queue_duration_s":2.63081,"scheduling_latency_s":2.630836,"redis_calls":13,"redis_duration_s":0.003934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001419,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002515,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024811,"completed_at":"2026-09-23T19:24:47.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.871729] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2f7d6fdf5d66c00d8082f1c","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2f7d6fdf5d66c00d8082f1c: start","job_status":"start","queue_duration_s":2.658739,"scheduling_latency_s":2.658764} gitlab # [ 437.884646] gitaly[892]: time="2026-09-23T19:24:48.610Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMKGBNMCEQ23YR7WS5A04 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:18.607" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:48.609" grpc.time_ms=1.469 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.897381] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 123 gitlab # [ 437.899891] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2f7d6fdf5d66c00d8082f1c","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2f7d6fdf5d66c00d8082f1c: done: 0.023127 sec","job_status":"done","queue_duration_s":2.658739,"scheduling_latency_s":2.658764,"redis_calls":13,"redis_duration_s":0.0031349999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023127,"completed_at":"2026-09-23T19:24:47.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.938946] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d11389eb41b62b9312fbca2","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d11389eb41b62b9312fbca2: start","job_status":"start","queue_duration_s":2.684828,"scheduling_latency_s":2.684879} gitlab # [ 437.950997] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d11389eb41b62b9312fbca2","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d11389eb41b62b9312fbca2: done: 0.039177 sec","job_status":"done","queue_duration_s":2.684828,"scheduling_latency_s":2.684879,"redis_calls":13,"redis_duration_s":0.004602,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001721,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039177,"completed_at":"2026-09-23T19:24:47.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.989777] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86a93d40c944d143cec08e9a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86a93d40c944d143cec08e9a: start","job_status":"start","queue_duration_s":2.726734,"scheduling_latency_s":2.726771} gitlab # [ 438.001747] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86a93d40c944d143cec08e9a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86a93d40c944d143cec08e9a: done: 0.029831 sec","job_status":"done","queue_duration_s":2.726734,"scheduling_latency_s":2.726771,"redis_calls":13,"redis_duration_s":0.003869,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001274,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002595,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029831,"completed_at":"2026-09-23T19:24:47.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.041769] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:47.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9169e2e293692665cf736ffd","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9169e2e293692665cf736ffd: start","job_status":"start","queue_duration_s":2.758666,"scheduling_latency_s":2.758701} gitlab # [ 438.054098] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9169e2e293692665cf736ffd","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9169e2e293692665cf736ffd: done: 0.024568 sec","job_status":"done","queue_duration_s":2.758666,"scheduling_latency_s":2.758701,"redis_calls":13,"redis_duration_s":0.003408,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001242,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002166,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024568,"completed_at":"2026-09-23T19:24:48.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.092840] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5efe3d2f0d1d88e9d237c79","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5efe3d2f0d1d88e9d237c79: start","job_status":"start","queue_duration_s":2.785146,"scheduling_latency_s":2.785174} gitlab # [ 438.105039] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5efe3d2f0d1d88e9d237c79","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5efe3d2f0d1d88e9d237c79: done: 0.019642 sec","job_status":"done","queue_duration_s":2.785146,"scheduling_latency_s":2.785174,"redis_calls":13,"redis_duration_s":0.002896,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019642,"completed_at":"2026-09-23T19:24:48.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.143569] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c659cf02772fac55aceeee1","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c659cf02772fac55aceeee1: start","job_status":"start","queue_duration_s":2.806827,"scheduling_latency_s":2.806853} gitlab # [ 438.156095] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c659cf02772fac55aceeee1","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c659cf02772fac55aceeee1: done: 0.018142 sec","job_status":"done","queue_duration_s":2.806827,"scheduling_latency_s":2.806853,"redis_calls":13,"redis_duration_s":0.003195,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001069,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018142,"completed_at":"2026-09-23T19:24:48.062Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.194288] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbee5885e7cd7d30ddbd83b0","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbee5885e7cd7d30ddbd83b0: start","job_status":"start","queue_duration_s":2.826516,"scheduling_latency_s":2.82654} gitlab # [ 438.206646] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbee5885e7cd7d30ddbd83b0","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbee5885e7cd7d30ddbd83b0: done: 0.016327 sec","job_status":"done","queue_duration_s":2.826516,"scheduling_latency_s":2.82654,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016327,"completed_at":"2026-09-23T19:24:48.080Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.247066] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"899ed71fc593cb7a52d7e9a4","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-899ed71fc593cb7a52d7e9a4: start","job_status":"start","queue_duration_s":2.844811,"scheduling_latency_s":2.844834} gitlab # [ 438.259551] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"899ed71fc593cb7a52d7e9a4","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-899ed71fc593cb7a52d7e9a4: done: 0.015937 sec","job_status":"done","queue_duration_s":2.844811,"scheduling_latency_s":2.844834,"redis_calls":13,"redis_duration_s":0.003029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001178,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001851,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015937,"completed_at":"2026-09-23T19:24:48.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.298115] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99eb058d1debee6b53912db7","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99eb058d1debee6b53912db7: start","job_status":"start","queue_duration_s":2.862255,"scheduling_latency_s":2.862279} gitlab # [ 438.310719] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99eb058d1debee6b53912db7","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99eb058d1debee6b53912db7: done: 0.015912 sec","job_status":"done","queue_duration_s":2.862255,"scheduling_latency_s":2.862279,"redis_calls":13,"redis_duration_s":0.002848,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015912,"completed_at":"2026-09-23T19:24:48.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.349246] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aa11fcabe88dee21c724f60","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aa11fcabe88dee21c724f60: start","job_status":"start","queue_duration_s":2.879629,"scheduling_latency_s":2.879647} gitlab # [ 438.361663] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aa11fcabe88dee21c724f60","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aa11fcabe88dee21c724f60: done: 0.015616 sec","job_status":"done","queue_duration_s":2.879629,"scheduling_latency_s":2.879647,"redis_calls":13,"redis_duration_s":0.002764,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015616,"completed_at":"2026-09-23T19:24:48.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.399814] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ba5f071d9535791760f04df","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ba5f071d9535791760f04df: start","job_status":"start","queue_duration_s":2.896644,"scheduling_latency_s":2.896663} gitlab # [ 438.412060] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ba5f071d9535791760f04df","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ba5f071d9535791760f04df: done: 0.015589 sec","job_status":"done","queue_duration_s":2.896644,"scheduling_latency_s":2.896663,"redis_calls":13,"redis_duration_s":0.00263,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001738,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015589,"completed_at":"2026-09-23T19:24:48.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.450329] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94d50569b7d23c8970b28b93","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94d50569b7d23c8970b28b93: start","job_status":"start","queue_duration_s":2.913779,"scheduling_latency_s":2.913798} gitlab # [ 438.463803] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94d50569b7d23c8970b28b93","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94d50569b7d23c8970b28b93: done: 0.01573 sec","job_status":"done","queue_duration_s":2.913779,"scheduling_latency_s":2.913798,"redis_calls":13,"redis_duration_s":0.0028510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01573,"completed_at":"2026-09-23T19:24:48.166Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.502165] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"644f6d9b86b7fbef2d63b7f3","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-644f6d9b86b7fbef2d63b7f3: start","job_status":"start","queue_duration_s":2.931006,"scheduling_latency_s":2.931025} gitlab # [ 438.514299] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"644f6d9b86b7fbef2d63b7f3","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-644f6d9b86b7fbef2d63b7f3: done: 0.015649 sec","job_status":"done","queue_duration_s":2.931006,"scheduling_latency_s":2.931025,"redis_calls":13,"redis_duration_s":0.002709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015649,"completed_at":"2026-09-23T19:24:48.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.551996] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"625c49d77e9c26062f4a6d4a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-625c49d77e9c26062f4a6d4a: start","job_status":"start","queue_duration_s":2.948179,"scheduling_latency_s":2.948198} gitlab # [ 438.563985] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"625c49d77e9c26062f4a6d4a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-625c49d77e9c26062f4a6d4a: done: 0.01555 sec","job_status":"done","queue_duration_s":2.948179,"scheduling_latency_s":2.948198,"redis_calls":13,"redis_duration_s":0.0029709999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00123,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001741,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01555,"completed_at":"2026-09-23T19:24:48.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.602423] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b86a9406872e17c7434758d0","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b86a9406872e17c7434758d0: start","job_status":"start","queue_duration_s":2.965615,"scheduling_latency_s":2.965635} gitlab # [ 438.614662] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b86a9406872e17c7434758d0","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b86a9406872e17c7434758d0: done: 0.015762 sec","job_status":"done","queue_duration_s":2.965615,"scheduling_latency_s":2.965635,"redis_calls":13,"redis_duration_s":0.002889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015762,"completed_at":"2026-09-23T19:24:48.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.653603] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a985cea6d1b39e4f4f946ef","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a985cea6d1b39e4f4f946ef: start","job_status":"start","queue_duration_s":2.982751,"scheduling_latency_s":2.982768} gitlab # [ 438.666995] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a985cea6d1b39e4f4f946ef","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a985cea6d1b39e4f4f946ef: done: 0.023461 sec","job_status":"done","queue_duration_s":2.982751,"scheduling_latency_s":2.982768,"redis_calls":13,"redis_duration_s":0.009413000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006589,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023461,"completed_at":"2026-09-23T19:24:48.243Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.706044] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c251ad4dabfa9a42b3de777","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c251ad4dabfa9a42b3de777: start","job_status":"start","queue_duration_s":3.007712,"scheduling_latency_s":3.007744} gitlab # [ 438.718539] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c251ad4dabfa9a42b3de777","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c251ad4dabfa9a42b3de777: done: 0.015554 sec","job_status":"done","queue_duration_s":3.007712,"scheduling_latency_s":3.007744,"redis_calls":13,"redis_duration_s":0.002659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001671,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015554,"completed_at":"2026-09-23T19:24:48.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.757391] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34edb879d8b8bfe0c9c8817e","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34edb879d8b8bfe0c9c8817e: start","job_status":"start","queue_duration_s":3.024703,"scheduling_latency_s":3.02472} gitlab # [ 438.769684] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34edb879d8b8bfe0c9c8817e","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34edb879d8b8bfe0c9c8817e: done: 0.015106 sec","job_status":"done","queue_duration_s":3.024703,"scheduling_latency_s":3.02472,"redis_calls":13,"redis_duration_s":0.002715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001822,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015106,"completed_at":"2026-09-23T19:24:48.277Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.808165] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"048dad836652b434f632e359","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-048dad836652b434f632e359: start","job_status":"start","queue_duration_s":3.041358,"scheduling_latency_s":3.041374} gitlab # [ 438.820397] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"048dad836652b434f632e359","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-048dad836652b434f632e359: done: 0.015941 sec","job_status":"done","queue_duration_s":3.041358,"scheduling_latency_s":3.041374,"redis_calls":13,"redis_duration_s":0.00337,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015941,"completed_at":"2026-09-23T19:24:48.294Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 438.859365] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d4277c12cdde6147c1564cf","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d4277c12cdde6147c1564cf: start","job_status":"start","queue_duration_s":3.058681,"scheduling_latency_s":3.058698} gitlab # [ 438.871497] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d4277c12cdde6147c1564cf","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d4277c12cdde6147c1564cf: done: 0.014887 sec","job_status":"done","queue_duration_s":3.058681,"scheduling_latency_s":3.058698,"redis_calls":13,"redis_duration_s":0.002669,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001772,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014887,"completed_at":"2026-09-23T19:24:48.310Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.908779] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d39a119d58cfea579721a292","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d39a119d58cfea579721a292: start","job_status":"start","queue_duration_s":3.075138,"scheduling_latency_s":3.075153} gitlab # [ 438.921207] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d39a119d58cfea579721a292","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d39a119d58cfea579721a292: done: 0.015408 sec","job_status":"done","queue_duration_s":3.075138,"scheduling_latency_s":3.075153,"redis_calls":13,"redis_duration_s":0.002776,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000992,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001784,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015408,"completed_at":"2026-09-23T19:24:48.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.960262] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ea8da84aa548c9a4d35affe","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ea8da84aa548c9a4d35affe: start","job_status":"start","queue_duration_s":3.091826,"scheduling_latency_s":3.091844} gitlab # [ 438.972317] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ea8da84aa548c9a4d35affe","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ea8da84aa548c9a4d35affe: done: 0.028689 sec","job_status":"done","queue_duration_s":3.091826,"scheduling_latency_s":3.091844,"redis_calls":13,"redis_duration_s":0.004842,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001267,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003575,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028689,"completed_at":"2026-09-23T19:24:48.357Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.011264] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46c71849545560a87d95b86a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46c71849545560a87d95b86a: start","job_status":"start","queue_duration_s":3.123805,"scheduling_latency_s":3.123852} gitlab # [ 439.023693] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46c71849545560a87d95b86a","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46c71849545560a87d95b86a: done: 0.036516 sec","job_status":"done","queue_duration_s":3.123805,"scheduling_latency_s":3.123852,"redis_calls":13,"redis_duration_s":0.0047469999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036516,"completed_at":"2026-09-23T19:24:48.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.063137] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dca64a04ed84f1cf0534eb4","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dca64a04ed84f1cf0534eb4: start","job_status":"start","queue_duration_s":3.162678,"scheduling_latency_s":3.162712} gitlab # [ 439.075619] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dca64a04ed84f1cf0534eb4","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dca64a04ed84f1cf0534eb4: done: 0.031022 sec","job_status":"done","queue_duration_s":3.162678,"scheduling_latency_s":3.162712,"redis_calls":13,"redis_duration_s":0.004181,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002705,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031022,"completed_at":"2026-09-23T19:24:48.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.27 seconds) gitlab # [ 439.115573] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca8d1148e595b3a59952236f","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca8d1148e595b3a59952236f: start","job_status":"start","queue_duration_s":3.195933,"scheduling_latency_s":3.195959} gitlab # [ 439.128276] gitaly[892]: time="2026-09-23T19:24:49.859Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMMPVBDWAYG3Y2AAWV8TC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:19.856" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:49.857" grpc.time_ms=1.326 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.140516] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 158 gitlab # [ 439.143060] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca8d1148e595b3a59952236f","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca8d1148e595b3a59952236f: done: 0.023385 sec","job_status":"done","queue_duration_s":3.195933,"scheduling_latency_s":3.195959,"redis_calls":13,"redis_duration_s":0.003653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001208,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002445,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023385,"completed_at":"2026-09-23T19:24:48.456Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.182554] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90db5eac3e47c5fe9bf2b219","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90db5eac3e47c5fe9bf2b219: start","job_status":"start","queue_duration_s":3.221117,"scheduling_latency_s":3.221141} gitlab # [ 439.195311] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90db5eac3e47c5fe9bf2b219","created_at":"2026-09-23T19:24:44.621Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-1690dbaedc5c4a8d","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1e37856db41a71d14cd151a261e1b79a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.237Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90db5eac3e47c5fe9bf2b219: done: 0.022774 sec","job_status":"done","queue_duration_s":3.221117,"scheduling_latency_s":3.221141,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022774,"completed_at":"2026-09-23T19:24:48.481Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.234345] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.482Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"c6b81ed375006c0fda09bc27","created_at":"2026-09-23T19:24:45.241Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01da1bf7eaf095bc6e8b38cfcdcd58e9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.315Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-c6b81ed375006c0fda09bc27: start","job_status":"start","queue_duration_s":3.167905,"scheduling_latency_s":3.167926} gitlab # [ 439.248180] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"c6b81ed375006c0fda09bc27","created_at":"2026-09-23T19:24:45.241Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01da1bf7eaf095bc6e8b38cfcdcd58e9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.315Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-c6b81ed375006c0fda09bc27: done: 0.085915 sec","job_status":"done","queue_duration_s":3.167905,"scheduling_latency_s":3.167926,"redis_calls":9,"redis_duration_s":0.0045590000000000006,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000745,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.085915,"completed_at":"2026-09-23T19:24:48.568Z","load_balancing_strategy":"primary","db_duration_s":0.019275,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.291531] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.572Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"64b342a8e495a90827d2028a","created_at":"2026-09-23T19:24:45.345Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04fb8154d6247cf10ad216c5119a9098","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.384Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-64b342a8e495a90827d2028a: start","job_status":"start","queue_duration_s":3.187377,"scheduling_latency_s":3.187426} gitlab # [ 439.304961] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.626Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"64b342a8e495a90827d2028a","created_at":"2026-09-23T19:24:45.345Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04fb8154d6247cf10ad216c5119a9098","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.384Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-64b342a8e495a90827d2028a: done: 0.054044 sec","job_status":"done","queue_duration_s":3.187377,"scheduling_latency_s":3.187426,"redis_calls":11,"redis_duration_s":0.004172,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000677,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001891,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0009869,"exclusive_lock_hold_duration_s":0.01218972,"duration_s":0.054044,"completed_at":"2026-09-23T19:24:48.625Z","load_balancing_strategy":"primary","db_duration_s":0.006855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.345523] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ba09494f56f6efff0c7f7ddb","created_at":"2026-09-23T19:24:45.417Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9ceda9ee62cd982a355dc7360195843","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.436Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ba09494f56f6efff0c7f7ddb: start","job_status":"start","queue_duration_s":3.191224,"scheduling_latency_s":3.191257} gitlab # [ 439.359606] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ba09494f56f6efff0c7f7ddb","created_at":"2026-09-23T19:24:45.417Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9ceda9ee62cd982a355dc7360195843","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.436Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ba09494f56f6efff0c7f7ddb: done: 0.034302 sec","job_status":"done","queue_duration_s":3.191224,"scheduling_latency_s":3.191257,"redis_calls":11,"redis_duration_s":0.002909,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000488,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00067362,"exclusive_lock_hold_duration_s":0.00433814,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.034302,"completed_at":"2026-09-23T19:24:48.662Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.401892] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.664Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"8f00c3502995d28fe1051aab","created_at":"2026-09-23T19:24:45.451Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"15137db9c91421160735cbd7e1a4e22a","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.466Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-8f00c3502995d28fe1051aab: start","job_status":"start","queue_duration_s":3.19851,"scheduling_latency_s":3.198544} gitlab # [ 439.415619] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.692Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"8f00c3502995d28fe1051aab","created_at":"2026-09-23T19:24:45.451Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"15137db9c91421160735cbd7e1a4e22a","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.466Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-8f00c3502995d28fe1051aab: done: 0.027559 sec","job_status":"done","queue_duration_s":3.19851,"scheduling_latency_s":3.198544,"redis_calls":10,"redis_duration_s":0.002717,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001578,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038518,"duration_s":0.027559,"completed_at":"2026-09-23T19:24:48.692Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.457631] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"381ea0245dc1e4c392aa4698","created_at":"2026-09-23T19:24:45.494Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.519Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-381ea0245dc1e4c392aa4698: start","job_status":"start","queue_duration_s":3.174625,"scheduling_latency_s":3.174646} gitlab # [ 439.471265] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.738Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"381ea0245dc1e4c392aa4698","created_at":"2026-09-23T19:24:45.494Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.519Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-381ea0245dc1e4c392aa4698: done: 0.044156 sec","job_status":"done","queue_duration_s":3.174625,"scheduling_latency_s":3.174646,"redis_calls":18,"redis_duration_s":0.004833,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001187,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000331,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002436,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044156,"completed_at":"2026-09-23T19:24:48.738Z","load_balancing_strategy":"primary","db_duration_s":0.003304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.514729] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.740Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"4e3719b0680b7b250f1323d3","created_at":"2026-09-23T19:24:45.548Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bee39d3d470bc7e772e59e73f4069e45","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:24:45.574Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-4e3719b0680b7b250f1323d3: start","job_status":"start","queue_duration_s":3.166,"scheduling_latency_s":3.16602} gitlab # [ 439.528131] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.753Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 439.531631] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"4e3719b0680b7b250f1323d3","created_at":"2026-09-23T19:24:45.548Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bee39d3d470bc7e772e59e73f4069e45","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:24:45.574Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-4e3719b0680b7b250f1323d3: done: 0.021977 sec","job_status":"done","queue_duration_s":3.166,"scheduling_latency_s":3.16602,"redis_calls":9,"redis_duration_s":0.0026650000000000003,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021977,"completed_at":"2026-09-23T19:24:48.762Z","load_balancing_strategy":"primary","db_duration_s":0.000689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.573031] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"e27aa2eb278cd1b4ef36a099","created_at":"2026-09-23T19:24:45.628Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc082a85e62baa3fd3de3c325809d386","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5dfc48484ff3623d3ff52edf287e601d22e2b83c1335126c2dbc7ea70ce18136","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.646Z","job_size_bytes":92,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-e27aa2eb278cd1b4ef36a099: start","job_status":"start","queue_duration_s":3.117729,"scheduling_latency_s":3.117748} gitlab # [ 439.587255] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"e27aa2eb278cd1b4ef36a099","created_at":"2026-09-23T19:24:45.628Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc082a85e62baa3fd3de3c325809d386","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5dfc48484ff3623d3ff52edf287e601d22e2b83c1335126c2dbc7ea70ce18136","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.646Z","job_size_bytes":92,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-e27aa2eb278cd1b4ef36a099: done: 0.075553 sec","job_status":"done","queue_duration_s":3.117729,"scheduling_latency_s":3.117748,"redis_calls":10,"redis_duration_s":0.002733,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000533,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001597,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000603,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040912,"duration_s":0.075553,"completed_at":"2026-09-23T19:24:48.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030702,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.629785] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4e3491062faa7c8b743c3f54","created_at":"2026-09-23T19:24:45.665Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88b0994a796a771dfd23abcbd4dfbfce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.689Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4e3491062faa7c8b743c3f54: start","job_status":"start","queue_duration_s":3.151419,"scheduling_latency_s":3.151438} gitlab # [ 439.644293] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.853Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 439.647594] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4e3491062faa7c8b743c3f54","created_at":"2026-09-23T19:24:45.665Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88b0994a796a771dfd23abcbd4dfbfce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.689Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4e3491062faa7c8b743c3f54: done: 0.01967 sec","job_status":"done","queue_duration_s":3.151419,"scheduling_latency_s":3.151438,"redis_calls":9,"redis_duration_s":0.002366,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000494,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01967,"completed_at":"2026-09-23T19:24:48.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.690380] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"919746c7bfc2209c011e3d8a","created_at":"2026-09-23T19:24:45.760Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64e57970a6d274fdaffbc387bdf58829","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:24:45.789Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-919746c7bfc2209c011e3d8a: start","job_status":"start","queue_duration_s":3.072886,"scheduling_latency_s":3.072903} gitlab # [ 439.703759] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"919746c7bfc2209c011e3d8a","created_at":"2026-09-23T19:24:45.760Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64e57970a6d274fdaffbc387bdf58829","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:24:45.789Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-919746c7bfc2209c011e3d8a: done: 0.034692 sec","job_status":"done","queue_duration_s":3.072886,"scheduling_latency_s":3.072903,"redis_calls":9,"redis_duration_s":0.002588,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000546,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034692,"completed_at":"2026-09-23T19:24:48.896Z","load_balancing_strategy":"primary","db_duration_s":0.007302,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.744367] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.898Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7676a205243451ae572c3c7a","created_at":"2026-09-23T19:24:45.820Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fdc23e537ee58eaf70ef1fc4ba4527e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.838Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7676a205243451ae572c3c7a: start","job_status":"start","queue_duration_s":3.060418,"scheduling_latency_s":3.060438} gitlab # [ 439.758070] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7676a205243451ae572c3c7a","created_at":"2026-09-23T19:24:45.820Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fdc23e537ee58eaf70ef1fc4ba4527e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.838Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7676a205243451ae572c3c7a: done: 0.026229 sec","job_status":"done","queue_duration_s":3.060418,"scheduling_latency_s":3.060438,"redis_calls":9,"redis_duration_s":0.0024129999999999998,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000547,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026229,"completed_at":"2026-09-23T19:24:48.924Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.798822] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"627145f465a1218272372be9","created_at":"2026-09-23T19:24:45.856Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6645a3e429dc013c8e42becf6bbe1cd7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.911Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-627145f465a1218272372be9: start","job_status":"start","queue_duration_s":3.015531,"scheduling_latency_s":3.01555} gitlab # [ 439.813074] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.938Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 439.816190] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.946Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"627145f465a1218272372be9","created_at":"2026-09-23T19:24:45.856Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6645a3e429dc013c8e42becf6bbe1cd7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.911Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-627145f465a1218272372be9: done: 0.020164 sec","job_status":"done","queue_duration_s":3.015531,"scheduling_latency_s":3.01555,"redis_calls":9,"redis_duration_s":0.002517,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000613,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020164,"completed_at":"2026-09-23T19:24:48.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.857328] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.948Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"b10a8df947697f0feae4ad6f","created_at":"2026-09-23T19:24:45.936Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c7cef8b1df91bed5bf4fce8a71e7adf","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.965Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-b10a8df947697f0feae4ad6f: start","job_status":"start","queue_duration_s":2.982346,"scheduling_latency_s":2.982364} gitlab # [ 439.871310] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"b10a8df947697f0feae4ad6f","created_at":"2026-09-23T19:24:45.936Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c7cef8b1df91bed5bf4fce8a71e7adf","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:45.965Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-b10a8df947697f0feae4ad6f: done: 0.026433 sec","job_status":"done","queue_duration_s":2.982346,"scheduling_latency_s":2.982364,"redis_calls":11,"redis_duration_s":0.003079,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00057,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00061976,"exclusive_lock_hold_duration_s":0.00637584,"duration_s":0.026433,"completed_at":"2026-09-23T19:24:48.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.914249] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.976Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1c73ccb19655cb3b88d99090","created_at":"2026-09-23T19:24:45.989Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40db0569552f5bad26073ff8b9930d04","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:46.008Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1c73ccb19655cb3b88d99090: start","job_status":"start","queue_duration_s":2.968216,"scheduling_latency_s":2.968236} gitlab # [ 439.927302] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.995Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1c73ccb19655cb3b88d99090","created_at":"2026-09-23T19:24:45.989Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40db0569552f5bad26073ff8b9930d04","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:46.008Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1c73ccb19655cb3b88d99090: done: 0.019607 sec","job_status":"done","queue_duration_s":2.968216,"scheduling_latency_s":2.968236,"redis_calls":11,"redis_duration_s":0.0027990000000000003,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000497,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000574,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001244,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019607,"completed_at":"2026-09-23T19:24:48.995Z","load_balancing_strategy":"primary","db_duration_s":0.000752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.969689] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:48.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"086b4054e56876e7c6e2bebe","created_at":"2026-09-23T19:24:46.018Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e513ddb49b47330c58782a281e074b88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:46.032Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-086b4054e56876e7c6e2bebe: start","job_status":"start","queue_duration_s":2.964748,"scheduling_latency_s":2.964767} gitlab # [ 439.984188] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"086b4054e56876e7c6e2bebe","created_at":"2026-09-23T19:24:46.018Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e513ddb49b47330c58782a281e074b88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:46.032Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-086b4054e56876e7c6e2bebe: done: 0.051977 sec","job_status":"done","queue_duration_s":2.964748,"scheduling_latency_s":2.964767,"redis_calls":11,"redis_duration_s":0.004358,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000611,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001371,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00081898,"exclusive_lock_hold_duration_s":0.02175972,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.051977,"completed_at":"2026-09-23T19:24:49.049Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.028305] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.051Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a6851a9e202d07d37daa0226","created_at":"2026-09-23T19:24:46.057Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e19f84aa39e86b3f76699069e5ae9a3","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:46.086Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a6851a9e202d07d37daa0226: start","job_status":"start","queue_duration_s":2.964982,"scheduling_latency_s":2.965001} gitlab # [ 440.042034] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.078Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a6851a9e202d07d37daa0226","created_at":"2026-09-23T19:24:46.057Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-a037d0f8f27f457e","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e19f84aa39e86b3f76699069e5ae9a3","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:46.086Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a6851a9e202d07d37daa0226: done: 0.027585 sec","job_status":"done","queue_duration_s":2.964982,"scheduling_latency_s":2.965001,"redis_calls":9,"redis_duration_s":0.002453,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000554,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027585,"completed_at":"2026-09-23T19:24:49.078Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.083544] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.080Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4c7d1ea48e475f92947480a8","created_at":"2026-09-23T19:24:48.714Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-f6423058f0d04909","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:48.729Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4c7d1ea48e475f92947480a8: start","job_status":"start","queue_duration_s":0.351004,"scheduling_latency_s":0.351025} gitlab # [ 440.095977] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.097Z","message":"No existing merge request to be cleaned up."} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 440.097752] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.112Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4c7d1ea48e475f92947480a8","created_at":"2026-09-23T19:24:48.714Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-f6423058f0d04909","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:48.729Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4c7d1ea48e475f92947480a8: done: 0.031787 sec","job_status":"done","queue_duration_s":0.351004,"scheduling_latency_s":0.351025,"redis_calls":12,"redis_duration_s":0.003346,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000594,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001146,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001606,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031787,"completed_at":"2026-09-23T19:24:49.112Z","load_balancing_strategy":"primary","db_duration_s":0.005943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.137827] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.113Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bf3a4d242beb271d90aed98c","created_at":"2026-09-23T19:24:48.714Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-f6423058f0d04909","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:48.729Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bf3a4d242beb271d90aed98c: start","job_status":"start","queue_duration_s":0.384274,"scheduling_latency_s":0.384292} gitlab # [ 440.148996] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.125Z","message":"No existing merge request to be cleaned up."} gitlab # [ 440.150512] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.134Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bf3a4d242beb271d90aed98c","created_at":"2026-09-23T19:24:48.714Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-f6423058f0d04909","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:48.729Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bf3a4d242beb271d90aed98c: done: 0.020473 sec","job_status":"done","queue_duration_s":0.384274,"scheduling_latency_s":0.384292,"redis_calls":10,"redis_duration_s":0.00269,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001575,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020473,"completed_at":"2026-09-23T19:24:49.134Z","load_balancing_strategy":"primary","db_duration_s":0.002633,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.186676] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.135Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0d0f32b4703e625be11792d2","created_at":"2026-09-23T19:24:48.714Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-f6423058f0d04909","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:48.729Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0d0f32b4703e625be11792d2: start","job_status":"start","queue_duration_s":0.40618,"scheduling_latency_s":0.406198} gitlab # [ 440.199134] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.146Z","message":"No existing merge request to be cleaned up."} gitlab # [ 440.200847] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.155Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0d0f32b4703e625be11792d2","created_at":"2026-09-23T19:24:48.714Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-f6423058f0d04909","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:48.729Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0d0f32b4703e625be11792d2: done: 0.019974 sec","job_status":"done","queue_duration_s":0.40618,"scheduling_latency_s":0.406198,"redis_calls":10,"redis_duration_s":0.002522,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001478,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019974,"completed_at":"2026-09-23T19:24:49.155Z","load_balancing_strategy":"primary","db_duration_s":0.002643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.239191] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.156Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4c763291d9e663fa4306daca","created_at":"2026-09-23T19:24:48.714Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-f6423058f0d04909","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:48.729Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4c763291d9e663fa4306daca: start","job_status":"start","queue_duration_s":0.427589,"scheduling_latency_s":0.427608} gitlab # [ 440.251764] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.168Z","message":"No existing merge request to be cleaned up."} gitlab # [ 440.253540] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:49.176Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4c763291d9e663fa4306daca","created_at":"2026-09-23T19:24:48.714Z","trace_propagation_headers":{"sentry-trace":"24a85ce75f3c4d009f34e3a2abcc8a20-f6423058f0d04909","baggage":"sentry-trace_id=24a85ce75f3c4d009f34e3a2abcc8a20,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"780818c2f7cff18d7866b33214b94485","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:24:48.729Z","job_size_bytes":2,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4c763291d9e663fa4306daca: done: 0.019987 sec","job_status":"done","queue_duration_s":0.427589,"scheduling_latency_s":0.427608,"redis_calls":10,"redis_duration_s":0.002537,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001462,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019987,"completed_at":"2026-09-23T19:24:49.176Z","load_balancing_strategy":"primary","db_duration_s":0.002498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.323925] gitaly[892]: time="2026-09-23T19:24:51.120Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMNYDENNMSGJPR60V1573 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:21.118" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:51.119" grpc.time_ms=0.999 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 440.369715] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 144 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.25 seconds) gitlab # [ 441.031801] nix-daemon[3757]: accepted connection from pid 4796, user nixbot gitlab # [ 441.239599] gitaly[892]: time="2026-09-23T19:24:52.036Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:22.033" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:52.034" grpc.time_ms=1.419 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.327269] gitaly[892]: time="2026-09-23T19:24:52.122Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:25:02.054" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.104" grpc.time_ms=17.752 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 441.409519] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.204Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"12a237566897a0c22ece6c41","created_at":"2026-09-23T19:24:52.192Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-4251874d9ecc4573","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.201Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-12a237566897a0c22ece6c41: start","job_status":"start","queue_duration_s":0.003063,"scheduling_latency_s":0.003135} gitlab # [ 441.451654] nixbot[3587]: {"ts": "2026-09-23T19:24:52.247Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 \"HTTP/1.1 201 Created\""} gitlab # [ 441.457585] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 325 gitlab # [ 441.521491] gitaly[892]: time="2026-09-23T19:24:52.318Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMQ4BFVR4BBAN44QRMHBG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:22.316" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:52.316" grpc.time_ms=1.231 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.19 seconds) gitlab # [ 441.560671] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 121 gitlab # [ 441.611160] gitaly[892]: time="2026-09-23T19:24:52.407Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMQ54THX606PQ9EJ4Z465 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:22.305" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:52.406" grpc.time_ms=1.302 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.658722] gitaly[892]: time="2026-09-23T19:24:52.455Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:52.454" grpc.time_ms=0.834 pid=892 span.kind=server system=grpc gitlab # [ 441.696153] gitaly[892]: time="2026-09-23T19:24:52.492Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VMQ54THX606PQ9EJ4Z465 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:25:02.078" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.478" grpc.time_ms=13.878 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.721039] gitaly[892]: time="2026-09-23T19:24:52.506Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:52.505" grpc.time_ms=0.517 pid=892 span.kind=server system=grpc gitlab # [ 441.751306] gitaly[892]: time="2026-09-23T19:24:52.548Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:24:52.547" grpc.time_ms=0.613 pid=892 span.kind=server system=grpc gitlab # [ 441.768311] gitaly[892]: time="2026-09-23T19:24:52.557Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:22.299" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.400" operations="[repack_refs]" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.784258] gitaly[892]: time="2026-09-23T19:24:52.562Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:22.299" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.400" pid=892 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-23 19:22:59.05252758 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 441.800740] gitaly[892]: time="2026-09-23T19:24:52.564Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 diskcache=59869136-2d66-4e4b-8a42-43a518212045 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:22.299" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.400" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.812681] gitaly[892]: time="2026-09-23T19:24:52.565Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=161 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7108 command.oublock=0 command.real_time_ms=155 command.spawn_token_fork_ms=1 command.system_time_ms=83 command.user_time_ms=78 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:22.299" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.400" grpc.time_ms=164.732 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.827816] gitaly[892]: time="2026-09-23T19:24:52.602Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:22.299" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.400" pid=892 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 441.842828] gitaly[892]: time="2026-09-23T19:24:52.602Z" level=info msg="optimized repository" clean_stale_data_time=39.6094ms clean_work_trees_time="158.8µs" commit_graph_time="1.26µs" correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.6µs" pid=892 prune_time=400ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="1.46µs" storage_name=default system=housekeeping total_time=39.80314ms gitlab # [ 441.849231] nixbot[3587]: {"ts": "2026-09-23T19:24:52.601Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 \"HTTP/1.1 201 Created\""} gitlab # [ 441.852067] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 339 gitlab # [ 441.854570] gitaly[892]: time="2026-09-23T19:24:52.602Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:25:22.299" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.400" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.893432] gitaly[892]: time="2026-09-23T19:24:52.690Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMQGFDP6KVSJRZXC9ED52 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:22.589" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:52.689" grpc.time_ms=0.635 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.906187] gitaly[892]: time="2026-09-23T19:24:52.695Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:25:22.293" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:52.693" grpc.time_ms=1.845 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.959384] gitaly[892]: time="2026-09-23T19:24:52.756Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M37VMQGFDP6KVSJRZXC9ED52 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:25:02.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.744" grpc.time_ms=11.15 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.978671] gitaly[892]: time="2026-09-23T19:24:52.770Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMPTH2NYYTHTGMWNPZ3P8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:22.069" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:52.769" grpc.time_ms=0.891 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.001020] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.797Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4991270133f02828a9f5b2b9","created_at":"2026-09-23T19:24:52.792Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"12a237566897a0c22ece6c41","job_size_bytes":3,"pid":3453,"job_status":"deduplicated","message":"PipelineProcessWorker JID-4991270133f02828a9f5b2b9: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 442.019627] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.801Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"12a237566897a0c22ece6c41","created_at":"2026-09-23T19:24:52.192Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-4251874d9ecc4573","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.201Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3453,"job_status":"rescheduled","message":"PipelineProcessWorker JID-12a237566897a0c22ece6c41: rescheduled"} gitlab # [ 442.037295] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.816Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"12a237566897a0c22ece6c41","created_at":"2026-09-23T19:24:52.192Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-4251874d9ecc4573","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.201Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-12a237566897a0c22ece6c41: done: 0.612326 sec","job_status":"done","queue_duration_s":0.003063,"scheduling_latency_s":0.003135,"gitaly_calls":3,"gitaly_duration_s":0.186642,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.034165,"redis_read_bytes":239,"redis_write_bytes":21786,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002606,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00279,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.006926,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14471,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.009156,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000555,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.006795,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002731,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.078,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.024,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.028,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.044,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.327301,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.50010074,"duration_s":0.612326,"completed_at":"2026-09-23T19:24:52.816Z","load_balancing_strategy":"primary","db_duration_s":0.081489,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 442.088540] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.819Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b98d0047214691d278abdc27","created_at":"2026-09-23T19:24:52.383Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.387Z","job_size_bytes":23,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b98d0047214691d278abdc27: start","job_status":"start","queue_duration_s":0.43224,"scheduling_latency_s":0.432288} gitlab # [ 442.104646] nixbot[3587]: {"ts": "2026-09-23T19:24:52.844Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 \"HTTP/1.1 201 Created\""} gitlab # [ 442.107301] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 217 gitlab # [ 442.109861] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.905Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b98d0047214691d278abdc27","created_at":"2026-09-23T19:24:52.383Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.387Z","job_size_bytes":23,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b98d0047214691d278abdc27: done: 0.085867 sec","job_status":"done","queue_duration_s":0.43224,"scheduling_latency_s":0.432288,"redis_calls":7,"redis_duration_s":0.005381,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004483,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.085867,"completed_at":"2026-09-23T19:24:52.905Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016423,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 442.151624] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.909Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"ff20490a37c918e741455b2d","created_at":"2026-09-23T19:24:52.578Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.582Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-ff20490a37c918e741455b2d: start","job_status":"start","queue_duration_s":0.326392,"scheduling_latency_s":0.326428} gitlab # [ 442.169309] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.958Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"ff20490a37c918e741455b2d","created_at":"2026-09-23T19:24:52.578Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.582Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-ff20490a37c918e741455b2d: done: 0.049534 sec","job_status":"done","queue_duration_s":0.326392,"scheduling_latency_s":0.326428,"redis_calls":7,"redis_duration_s":0.004043,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002618,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001425,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049534,"completed_at":"2026-09-23T19:24:52.958Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007873,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.211266] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:52.962Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9c4c3f813655e3deda43b5e7","created_at":"2026-09-23T19:24:52.594Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.598Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9c4c3f813655e3deda43b5e7: start","job_status":"start","queue_duration_s":0.363755,"scheduling_latency_s":0.363791} gitlab # [ 442.226974] gitaly[892]: time="2026-09-23T19:24:52.950Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMQQS126CPZ6ZHCWWP2K9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:22.047" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:52.948" grpc.time_ms=1.197 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.239574] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.004Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9c4c3f813655e3deda43b5e7","created_at":"2026-09-23T19:24:52.594Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.598Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9c4c3f813655e3deda43b5e7: done: 0.042696 sec","job_status":"done","queue_duration_s":0.363755,"scheduling_latency_s":0.363791,"redis_calls":7,"redis_duration_s":0.00252,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00205,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042696,"completed_at":"2026-09-23T19:24:53.004Z","load_balancing_strategy":"primary","db_duration_s":0.00585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.280955] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.006Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"28669d3d3f936ca45bcc5829","created_at":"2026-09-23T19:24:52.619Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.623Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-28669d3d3f936ca45bcc5829: start","job_status":"start","queue_duration_s":0.383637,"scheduling_latency_s":0.38367} gitlab # [ 442.296245] gitaly[892]: time="2026-09-23T19:24:53.002Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=6 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VMQQS126CPZ6ZHCWWP2K9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:25:02.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:24:52.987" grpc.time_ms=14.877 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.309515] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.044Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"28669d3d3f936ca45bcc5829","created_at":"2026-09-23T19:24:52.619Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.623Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-28669d3d3f936ca45bcc5829: done: 0.037973 sec","job_status":"done","queue_duration_s":0.383637,"scheduling_latency_s":0.38367,"redis_calls":10,"redis_duration_s":0.001907,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000504,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037973,"completed_at":"2026-09-23T19:24:53.044Z","load_balancing_strategy":"primary","db_duration_s":0.00574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.352124] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.046Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d5302bea84042efc7b38712e","created_at":"2026-09-23T19:24:52.624Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.632Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d5302bea84042efc7b38712e: start","job_status":"start","queue_duration_s":0.413948,"scheduling_latency_s":0.413975} gitlab # [ 442.369434] nixbot[3587]: {"ts": "2026-09-23T19:24:53.097Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 \"HTTP/1.1 201 Created\""} gitlab # [ 442.372102] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/23:19:24:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e6cb306e04578deaef228fc2a54c36a018a70635 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 238 gitlab # [ 442.374549] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.081Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d5302bea84042efc7b38712e","created_at":"2026-09-23T19:24:52.624Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.632Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d5302bea84042efc7b38712e: done: 0.03503 sec","job_status":"done","queue_duration_s":0.413948,"scheduling_latency_s":0.413975,"redis_calls":7,"redis_duration_s":0.001909,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001471,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03503,"completed_at":"2026-09-23T19:24:53.081Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005831,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.416859] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.083Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f22f8a5baf6e394f54273b3c","created_at":"2026-09-23T19:24:52.633Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.639Z","job_size_bytes":19,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f22f8a5baf6e394f54273b3c: start","job_status":"start","queue_duration_s":0.443728,"scheduling_latency_s":0.443765} gitlab # [ 442.432802] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.128Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f22f8a5baf6e394f54273b3c","created_at":"2026-09-23T19:24:52.633Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.639Z","job_size_bytes":19,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f22f8a5baf6e394f54273b3c: done: 0.045436 sec","job_status":"done","queue_duration_s":0.443728,"scheduling_latency_s":0.443765,"redis_calls":8,"redis_duration_s":0.002355,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001828,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045436,"completed_at":"2026-09-23T19:24:53.128Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005199,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.474514] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.131Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d6f822301c51aa0e29efa183","created_at":"2026-09-23T19:24:52.642Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.647Z","job_size_bytes":66,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d6f822301c51aa0e29efa183: start","job_status":"start","queue_duration_s":0.484631,"scheduling_latency_s":0.484679} gitlab # [ 442.491602] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.175Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d6f822301c51aa0e29efa183","created_at":"2026-09-23T19:24:52.642Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.647Z","job_size_bytes":66,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d6f822301c51aa0e29efa183: done: 0.044286 sec","job_status":"done","queue_duration_s":0.484631,"scheduling_latency_s":0.484679,"redis_calls":7,"redis_duration_s":0.001901,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044286,"completed_at":"2026-09-23T19:24:53.175Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses gitlab # [ 442.652192] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.443Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"348fca880df958be01dad71f","created_at":"2026-09-23T19:24:52.650Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.655Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-348fca880df958be01dad71f: start","job_status":"start","queue_duration_s":0.787628,"scheduling_latency_s":0.78768} gitlab # [ 442.703578] gitaly[892]: time="2026-09-23T19:24:53.499Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VMR9J3QCEFTTG8MZT789V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:25:23.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:24:53.497" grpc.time_ms=1.928 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e6cb306e04578deaef228fc2a54c36a018a70635/statuses, in 0.20 seconds) (finished: subtest: eval and build post commit statuses, in 11.14 seconds) (finished: run the VM test script, in 514.44 seconds) gitlab # [ 442.727060] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.502Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"348fca880df958be01dad71f","created_at":"2026-09-23T19:24:52.650Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.655Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-348fca880df958be01dad71f: done: 0.064473 sec","job_status":"done","queue_duration_s":0.787628,"scheduling_latency_s":0.78768,"redis_calls":7,"redis_duration_s":0.002661,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000841,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.064473,"completed_at":"2026-09-23T19:24:53.502Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007267,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} test script finished in 514.49s cleanup gitlab # [ 442.791672] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.504Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"0584890504033f8fe7fbe8cc","created_at":"2026-09-23T19:24:52.664Z","trace_propagation_headers":{"sentry-trace":"650c9c80504a4f4c9c6ef446cf065e97-51291645c0404db1","baggage":"sentry-trace_id=650c9c80504a4f4c9c6ef446cf065e97,sentry-release=Unknown"},"correlation_id":"01M37VMPTH2NYYTHTGMWNPZ3P8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-23T19:24:52.668Z","job_size_bytes":3,"pid":3453,"sidekiq_tid":"m8lx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0584890504033f8fe7fbe8cc: start","job_status":"start","queue_duration_s":0.836195,"scheduling_latency_s":0.836215} kill QemuMachine (pid 46) gitlab # [ 442.814024] sidekiq-cluster[3453]: {"severity":"INFO","time":"2026-09-23T19:24:53.544Z","retry":25,"queue":"default","version"qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.73 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-23T19:23:13.718Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-23T19:24:42.164223Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] [] [{'id': 7, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:24:42.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 7, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:24:42.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 7, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:24:42.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 7, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:24:42.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 7, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:24:42.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 7, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:24:42.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 7, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:24:42.548Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 7, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-23T19:24:42.548Z', 'started_at': None, 'finished_at': '2026-09-23T19:24:52.172Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 8, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-23T19:24:52.554Z', 'started_at': None, 'finished_at': '2026-09-23T19:24:53.042Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 9, 'sha': 'e6cb306e04578deaef228fc2a54c36a018a70635', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-23T19:24:52.801Z', 'started_at': None, 'finished_at': '2026-09-23T19:24:52.796Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]