tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.1cDkpTe6vf', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 69.15 seconds) gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for unit setup-gitlab.service gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab: waiting for the VM to finish booting gitlab # Filesystem UUID: d31ce65a-c197-46b2-a916-9f9d6830db7f gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] KASLR enabled gitlab # [ 0.000000] random: crng init done gitlab # [ 0.000000] Machine model: linux,dummy-virt gitlab # [ 0.000000] efi: UEFI not found. gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] gitlab # [ 0.000000] Zone ranges: gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] gitlab # [ 0.000000] DMA32 empty gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.000000] Device empty gitlab # [ 0.000000] Movable zone start for each node gitlab # [ 0.000000] Early memory node ranges gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 gitlab # [ 0.000000] psci: probing for conduit method from DT. gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs gitlab # [ 0.000000] psci: Trusted OS migration not required gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface gitlab # [ 0.000000] CPU features: detected: Spectre-v4 gitlab # [ 0.000000] CPU features: detected: Spectre-BHB gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 gitlab # [ 0.000000] alternatives: applying boot alternatives gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fdfalam2a4qbn0d922kdr5380mx3zxnp-nixos-system-gitlab-test/init regInfo=/nix/store/rjpbp5dnf4r4mjbzx244j6fvmq0fjchb-closure-info/registration console=ttyAMA0,115200n8 console=tty0 gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/rjpbp5dnf4r4mjbzx244j6fvmq0fjchb-closure-info/registration", will be passed to user space. gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.000000] software IO TLB: area num 4. gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) gitlab # [ 0.000000] Fallback order for Node 0: 0 gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 gitlab # [ 0.000000] Policy zone: Normal gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.000000] allocated 12582912 bytes of page_ext gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. gitlab # [ 0.000000] rcu: RCU event tracing is enabled. gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 gitlab # [ 0.000000] GICv3: 256 SPIs implemented gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f20000 gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f30000 gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns gitlab # [ 0.000029] arm-pv: using stolen time PV gitlab # [ 0.000380] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.000589] Console: colour dummy device 80x25 gitlab # [ 0.000595] printk: legacy console [tty0] enabled gitlab # [ 0.000772] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) gitlab # [ 0.000779] pid_max: default: 32768 minimum: 301 gitlab # [ 0.000857] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.000989] landlock: Up and running. gitlab # [ 0.000991] Yama: becoming mindful. gitlab # [ 0.001410] LSM support for eBPF active gitlab # [ 0.001585] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.001697] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.002985] cacheinfo: Unable to detect cache hierarchy for CPU 0 gitlab # [ 0.003632] rcu: Hierarchical SRCU implementation. gitlab # [ 0.003636] rcu: Max phase no-delay instances is 1000. gitlab # [ 0.003817] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.004856] fsl-mc MSI: its@8080000 domain created gitlab # [ 0.004945] EFI services will not be available. gitlab # [ 0.005117] smp: Bringing up secondary CPUs ... gitlab # [ 0.005751] Detected PIPT I-cache on CPU1 gitlab # [ 0.005850] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 gitlab # [ 0.005981] GICv3: CPU1: using allocated LPI pending table @0x0000000100f40000 gitlab # [ 0.006109] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] gitlab # [ 0.007187] Detected PIPT I-cache on CPU2 gitlab # [ 0.007296] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 gitlab # [ 0.007427] GICv3: CPU2: using allocated LPI pending table @0x0000000100f50000 gitlab # [ 0.007555] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] gitlab # [ 0.008614] Detected PIPT I-cache on CPU3 gitlab # [ 0.008733] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 gitlab # [ 0.008864] GICv3: CPU3: using allocated LPI pending table @0x0000000100f60000 gitlab # [ 0.008994] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] gitlab # [ 0.009523] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.009550] SMP: Total of 4 processors activated. gitlab # [ 0.009552] CPU: All CPU(s) started at EL1 gitlab # [ 0.009560] CPU features: detected: Branch Target Identification gitlab # [ 0.009563] CPU features: detected: ARMv8.4 Translation Table Level gitlab # [ 0.009566] CPU features: detected: Instruction cache invalidation not required for I/D coherence gitlab # [ 0.009569] CPU features: detected: Data cache clean to the PoU not required for I/D coherence gitlab # [ 0.009573] CPU features: detected: Common not Private translations gitlab # [ 0.009576] CPU features: detected: CRC32 instructions gitlab # [ 0.009578] CPU features: detected: Data cache clean to Point of Deep Persistence gitlab # [ 0.009581] CPU features: detected: Data cache clean to Point of Persistence gitlab # [ 0.009584] CPU features: detected: Data independent timing control (DIT) gitlab # [ 0.009587] CPU features: detected: E0PD gitlab # [ 0.009589] CPU features: detected: Enhanced Counter Virtualization gitlab # [ 0.009592] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) gitlab # [ 0.009595] CPU features: detected: Enhanced Virtualization Traps gitlab # [ 0.009598] CPU features: detected: Fine Grained Traps gitlab # [ 0.009600] CPU features: detected: Generic authentication (architected QARMA5 algorithm) gitlab # [ 0.009604] CPU features: detected: RCpc load-acquire (LDAPR) gitlab # [ 0.009607] CPU features: detected: LSE atomic instructions gitlab # [ 0.009609] CPU features: detected: Privileged Access Never gitlab # [ 0.009612] CPU features: detected: PMUv3 gitlab # [ 0.009614] CPU features: detected: RAS Extension Support gitlab # [ 0.009616] CPU features: detected: RASv1p1 Extension Support gitlab # [ 0.009619] CPU features: detected: Random Number Generator gitlab # [ 0.009621] CPU features: detected: Speculation barrier (SB) gitlab # [ 0.009623] CPU features: detected: Stage-2 Force Write-Back gitlab # [ 0.009626] CPU features: detected: TLB range maintenance instructions gitlab # [ 0.009629] CPU features: detected: Speculative Store Bypassing Safe (SSBS) gitlab # [ 0.009758] alternatives: applying system-wide alternatives gitlab # [ 0.012609] CPU features: detected: BBM Level 2 without TLB conflict abort gitlab # [ 0.012836] Memory: 5963816K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) gitlab # [ 0.014313] devtmpfs: initialized gitlab # [ 0.018252] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.018317] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.018483] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL gitlab # [ 0.018488] 0 pages in range for non-PLT usage gitlab # [ 0.018489] 508288 pages in range for PLT usage gitlab # [ 0.018615] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.019428] DMI not present or invalid. gitlab # [ 0.023139] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.025568] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.025840] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.027700] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.027721] audit: initializing netlink subsys (disabled) gitlab # [ 0.028044] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.028468] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.028471] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.028475] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.028478] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.028480] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.028570] cpuidle: using governor ladder gitlab # [ 0.028633] cpuidle: using governor menu gitlab # [ 0.028845] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. gitlab # [ 0.029053] ASID allocator initialised with 65536 entries gitlab # [ 0.030124] Serial: AMBA PL011 UART driver gitlab # [ 0.035123] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 gitlab # [ 0.035467] printk: console [ttyAMA0] enabled gitlab # [ 0.046999] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.047020] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.047029] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages gitlab # [ 0.047037] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page gitlab # [ 0.047046] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.047053] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.047062] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages gitlab # [ 0.047070] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page gitlab # [ 0.058865] fbcon: Taking over console gitlab # [ 0.058875] ACPI: Interpreter disabled. gitlab # [ 0.059862] iommu: Default domain type: Translated gitlab # [ 0.059878] iommu: DMA domain TLB invalidation policy: strict mode gitlab # [ 0.069507] SCSI subsystem initialized gitlab # [ 0.069923] usbcore: registered new interface driver usbfs gitlab # [ 0.069963] usbcore: registered new interface driver hub gitlab # [ 0.069978] usbcore: registered new device driver usb gitlab # [ 0.070238] pps_core: LinuxPPS API ver. 1 registered gitlab # [ 0.070242] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti gitlab # [ 0.070250] PTP clock support registered gitlab # [ 0.070293] EDAC MC: Ver: 3.0.0 gitlab # [ 0.070819] scmi_core: SCMI protocol bus registered gitlab # [ 0.071698] FPGA manager framework gitlab # [ 0.072489] vgaarb: loaded gitlab # [ 0.074688] clocksource: Switched to clocksource arch_sys_counter gitlab # [ 0.076125] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.076186] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.077034] netfs: FS-Cache loaded gitlab # [ 0.077329] pnp: PnP ACPI: disabled gitlab # [ 0.086631] NET: Registered PF_INET protocol family gitlab # [ 0.089416] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.149976] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.150045] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.150106] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.150512] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.151493] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.151631] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.151704] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.152226] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.152486] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.152533] NET: Registered PF_XDP protocol family gitlab # [ 0.152554] PCI: CLS 0 bytes, default 64 gitlab # [ 0.152898] Trying to unpack rootfs image as initramfs... gitlab # [ 0.163202] kvm [1]: HYP mode not available gitlab # [ 0.235377] Initialise system trusted keyrings gitlab # [ 0.238925] workingset: timestamp_bits=42 max_order=21 bucket_order=0 gitlab # [ 0.243565] squashfs: version 4.0 (2009/01/31) Phillip Lougher gitlab # [ 0.243738] 9p: Installing v9fs 9p2000 file system support gitlab # [ 0.267648] Key type asymmetric registered gitlab # [ 0.267678] Asymmetric key parser 'x509' registered gitlab # [ 0.267848] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) gitlab # [ 0.268112] io scheduler mq-deadline registered gitlab # [ 0.268120] io scheduler kyber registered gitlab # [ 0.277617] pl061_gpio 9030000.pl061: PL061 GPIO chip registered gitlab # [ 0.278766] ledtrig-cpu: registered to indicate activity on CPUs gitlab # [ 0.279332] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: gitlab # [ 0.279352] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 gitlab # [ 0.279366] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 gitlab # [ 0.279376] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 gitlab # [ 0.279416] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits gitlab # [ 0.279445] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] gitlab # [ 0.279560] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 gitlab # [ 0.279571] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.279577] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] gitlab # [ 0.279582] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] gitlab # [ 0.279587] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] gitlab # [ 0.279677] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.280586] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.280961] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.280981] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.281032] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.281052] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.281923] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.282329] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.282349] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.282406] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.283328] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.283734] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.283754] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.283807] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.284696] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.285088] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.285108] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.285160] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.286054] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.286441] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.286460] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.286517] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.287405] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.287777] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.287797] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.287847] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.288751] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.289113] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.289132] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.289183] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.289203] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.290120] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint gitlab # [ 0.290496] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.290548] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.291397] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.291769] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.291820] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.292581] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint gitlab # [ 0.292939] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] gitlab # [ 0.293392] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.293816] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] gitlab # [ 0.293836] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.293883] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.294770] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.295110] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.295130] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.295181] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.296246] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned gitlab # [ 0.296256] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned gitlab # [ 0.296262] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned gitlab # [ 0.296339] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned gitlab # [ 0.296418] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned gitlab # [ 0.296505] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned gitlab # [ 0.296584] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned gitlab # [ 0.296662] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned gitlab # [ 0.296737] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned gitlab # [ 0.296820] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned gitlab # [ 0.296899] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned gitlab # [ 0.296981] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned gitlab # [ 0.297099] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned gitlab # [ 0.297172] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned gitlab # [ 0.297202] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned gitlab # [ 0.297231] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned gitlab # [ 0.297262] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned gitlab # [ 0.297291] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned gitlab # [ 0.297320] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned gitlab # [ 0.297350] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned gitlab # [ 0.297380] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned gitlab # [ 0.297411] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned gitlab # [ 0.297442] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned gitlab # [ 0.297476] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned gitlab # [ 0.297507] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned gitlab # [ 0.297536] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned gitlab # [ 0.297573] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned gitlab # [ 0.297602] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned gitlab # [ 0.297631] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned gitlab # [ 0.297661] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned gitlab # [ 0.297691] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned gitlab # [ 0.297721] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned gitlab # [ 0.297752] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned gitlab # [ 0.297783] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned gitlab # [ 0.297819] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] gitlab # [ 0.297827] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] gitlab # [ 0.297833] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] gitlab # [ 0.300110] pci 0000:00:0a.0: enabling device (0000 -> 0002) gitlab # [ 0.318977] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) gitlab # [ 0.322042] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) gitlab # [ 0.325381] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) gitlab # [ 0.328657] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) gitlab # [ 0.334218] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) gitlab # [ 0.337591] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) gitlab # [ 0.340754] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) gitlab # [ 0.345545] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) gitlab # [ 0.349751] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) gitlab # [ 0.352339] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) gitlab # [ 0.355423] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) gitlab # [ 0.367130] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.369722] msm_serial: driver initialized gitlab # [ 0.369983] SuperH (H)SCI(F) driver initialized gitlab # [ 0.370054] STM32 USART driver initialized gitlab # [ 0.404432] loop: module loaded gitlab # [ 0.404734] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 0.407457] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 0.410037] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 0.412599] virtio_blk virtio5: [vdb] 12284848 512-byte logical blocks (6.29 GB/5.86 GiB) gitlab # [ 0.415483] megasas: 07.734.00.00-rc1 gitlab # [ 0.416458] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] gitlab # [ 0.419343] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.419362] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.421618] Using buffer write method gitlab # [ 0.421665] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] gitlab # [ 0.424282] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.424304] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.426900] Using buffer write method gitlab # [ 0.426919] Concatenating MTD devices: gitlab # [ 0.426923] (0): "0.flash" gitlab # [ 0.426927] (1): "0.flash" gitlab # [ 0.426930] into device "0.flash" gitlab # [ 0.597579] Freeing initrd memory: 27184K gitlab # [ 0.610592] tun: Universal TUN/TAP device driver, 1.6 gitlab # [ 0.614682] thunder_xcv, ver 1.0 gitlab # [ 0.614768] thunder_bgx, ver 1.0 gitlab # [ 0.614791] nicpf, ver 1.0 gitlab # [ 0.615337] e1000: Intel(R) PRO/1000 Network Driver gitlab # [ 0.615342] e1000: Copyright (c) 1999-2006 Intel Corporation. gitlab # [ 0.615367] e1000e: Intel(R) PRO/1000 Network Driver gitlab # [ 0.615373] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. gitlab # [ 0.615397] igb: Intel(R) Gigabit Ethernet Network Driver gitlab # [ 0.615400] igb: Copyright (c) 2007-2014 Intel Corporation. gitlab # [ 0.615420] igbvf: Intel(R) Gigabit Virtual Function Network Driver gitlab # [ 0.615424] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. gitlab # [ 0.615553] sky2: driver version 1.30 gitlab # [ 0.617150] usbcore: registered new interface driver usb-storage gitlab # [ 0.617338] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.617352] usbserial: USB Serial support registered for generic gitlab # [ 0.617926] hv_vmbus: registering driver hyperv_keyboard gitlab # [ 0.618660] ehci-pci 0000:00:0a.0: EHCI Host Controller gitlab # [ 0.618784] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 gitlab # [ 0.619017] rtc-pl031 9010000.pl031: registered as rtc0 gitlab # [ 0.619043] rtc-pl031 9010000.pl031: setting system clock to 2026-09-05T11:53:47 UTC (1788609227) gitlab # [ 0.619334] i2c_dev: i2c /dev entries driver gitlab # [ 0.619469] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 gitlab # [ 0.621999] sdhci: Secure Digital Host Controller Interface driver gitlab # [ 0.622005] sdhci: Copyright(c) Pierre Ossman gitlab # [ 0.622259] Synopsys Designware Multimedia Card Interface Driver gitlab # [ 0.622615] sdhci-pltfm: SDHCI platform and OF driver helper gitlab # [ 0.624227] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 0.624555] usbcore: registered new interface driver usbhid gitlab # [ 0.624560] usbhid: USB HID core driver gitlab # [ 0.630790] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 gitlab # [ 0.632221] hub 1-0:1.0: USB hub found gitlab # [ 0.632334] hub 1-0:1.0: 6 ports detected gitlab # [ 0.674450] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available gitlab # [ 0.677547] drop_monitor: Initializing network drop monitor service gitlab # [ 0.677821] NET: Registered PF_INET6 protocol family gitlab # [ 0.679533] Segment Routing with IPv6 gitlab # [ 0.679575] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.679784] NET: Registered PF_PACKET protocol family gitlab # [ 0.680094] 9pnet: Installing 9P2000 support gitlab # [ 0.684488] Key type dns_resolver registered gitlab # [ 0.703973] registered taskstats version 1 gitlab # [ 0.704493] Loading compiled-in X.509 certificates gitlab # [ 0.731867] Demotion targets for Node 0: null gitlab # [ 0.732173] Key type .fscrypt registered gitlab # [ 0.732183] Key type fscrypt-provisioning registered gitlab # [ 0.732424] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.732452] ima: Allocated hash algorithm: sha1 gitlab # [ 0.732489] ima: No architecture policies found gitlab # [ 0.733815] input: gpio-keys as /devices/platform/gpio-keys/input/input0 gitlab # [ 0.772855] clk: Disabling unused clocks gitlab # [ 0.772898] PM: genpd: Disabling unused power domains gitlab # [ 0.781929] Freeing unused kernel memory: 4736K gitlab # [ 0.782313] Run /init as init process gitlab # [ 0.825891] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 0.878819] usb 1-1: new high-speed USB device number 2 using ehci-pci gitlab # [ 1.035269] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 gitlab # [ 1.123140] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 gitlab # [ 1.160642] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.160705] systemd[1]: Detected virtualization qemu. gitlab # [ 1.160779] systemd[1]: Detected architecture arm64. gitlab # [ 1.160796] systemd[1]: Running in initrd. gitlab # [ 1.161650] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.161904] systemd[1]: Hostname set to . gitlab # [ 1.250719] usb 1-2: new high-speed USB device number 3 using ehci-pci gitlab # [ 1.410862] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 gitlab # [ 1.411198] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 gitlab # [ 1.437952] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.526165] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.555095] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.555457] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.555513] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.555533] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.555568] systemd[1]: Reached target Path Units. gitlab # [ 1.555596] systemd[1]: Reached target Slice Units. gitlab # [ 1.555622] systemd[1]: Reached target Swaps. gitlab # [ 1.555647] systemd[1]: Reached target Timer Units. gitlab # [ 1.555896] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.556152] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.556394] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.556661] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.556800] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.556832] systemd[1]: Reached target Socket Units. gitlab # [ 1.559398] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.561958] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.562061] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 1.564894] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 1.595067] systemd[1]: Starting Journal Service... gitlab # [ 1.599312] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.599591] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.602899] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.607355] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.609918] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.611224] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.612354] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 1.616136] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.645163] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 gitlab # [ 1.645437] [drm] features: -virgl +edid -resource_blob -host_visible gitlab # [ 1.645455] [drm] features: -context_init gitlab # [ 1.647537] [drm] number of scanouts: 1 gitlab # [ 1.647564] [drm] number of cap sets: 0 gitlab # [ 1.647788] systemd-journald[95]: Collecting audit messages is disabled. gitlab # [ 1.648884] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.653622] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.653893] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic gitlab # [ 1.653910] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 gitlab # [ 1.662925] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 1.664415] Console: switching to colour frame buffer device 160x50 gitlab # [ 1.672188] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device gitlab # [ 1.677646] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 1.707246] systemd[1]: Finished Load Kernel Modules. gitlab # [ 1.712457] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.723998] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.724360] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.724419] systemd[1]: Reached target Local File Systems. gitlab # [ 1.730962] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.733056] systemd[1]: Started Journal Service. gitlab # [ 1.731861] systemd-modules-load[96]: Using 4 probe threads gitlab # [ 1.734992] systemd-modules-load[96]: Module 'virtio_balloon' is built in gitlab # [ 1.736780] systemd-modules-load[96]: Module 'virtio_console' is built in gitlab # [ 1.738627] systemd-modules-load[96]: Module 'virtio_rng' is built in gitlab # [ 1.739842] systemd-modules-load[96]: Inserted module 'dm_mod' gitlab # [ 1.742169] systemd-modules-load[96]: Inserted module 'virtio_gpu' gitlab # [ 1.743203] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.747320] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.748762] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.784354] systemd-udevd[107]: Using default interface naming scheme 'v261'. gitlab # [ 1.803861] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.831748] systemd[1]: Starting Virtual Console Setup... gitlab # [ 1.904163] systemd-vconsole-setup[129]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 1.909147] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.260423] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.261802] systemd[1]: Reached target System Initialization. gitlab # [ 2.262663] systemd[1]: Reached target Basic System. gitlab # [ 2.458997] (udev-worker)[136]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.462861] (udev-worker)[142]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.468240] (udev-worker)[142]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.503819] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.511728] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.514646] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.527608] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.559142] systemd-fsck[151]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.563183] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.604291] systemd[1]: Mounting /sysroot... gitlab # [ 2.643418] EXT4-fs (vda): mounted filesystem d31ce65a-c197-46b2-a916-9f9d6830db7f r/w with ordered data mode. Quota mode: none. gitlab # [ 2.643932] systemd[1]: Mounted /sysroot. gitlab # [ 2.644863] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.650062] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.653835] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.664817] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.674781] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 2.677189] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 2.700454] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 2.714561] systemd-sysroot-fstab-check[165]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 2.726148] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 2.727227] systemd[1]: Mounted /sysroot/run. gitlab # [ 2.744543] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.746737] systemd[1]: Reload requested from client PID 165 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 2.749052] systemd[1]: Reloading... gitlab # [ 2.754759] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 2.848475] systemd[1]: Reloading finished in 101 ms. gitlab # [ 2.912587] systemd-sysroot-fstab-check[165]: Requesting initrd-fs.target/start/replace... gitlab # [ 2.915594] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 2.916726] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 2.917450] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 2.918155] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 2.919138] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 2.920160] systemd-sysroot-fstab-check[165]: Requesting swap.target/start/replace... gitlab # [ 2.922710] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.924725] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 2.926012] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 2.927112] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 2.948087] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 2.949383] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 2.950611] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 2.969882] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 2.970634] systemd[1]: Reached target Initrd File Systems. gitlab # [ 2.971407] systemd[1]: Starting Find NixOS closure... gitlab # [ 2.979413] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 2.986053] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 2.992592] systemd[1]: Finished Find NixOS closure. gitlab # [ 2.993372] systemd[1]: Reached target Initrd Default Target. gitlab # [ 2.994207] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.025281] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.026270] systemd[1]: Stopped target Basic System. gitlab # [ 3.027005] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.027783] systemd[1]: Stopped target Path Units. gitlab # [ 3.028527] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.029560] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.030626] systemd[1]: Stopped target Slice Units. gitlab # [ 3.031330] systemd[1]: Stopped target Socket Units. gitlab # [ 3.032050] systemd[1]: Stopped target System Initialization. gitlab # [ 3.032849] systemd[1]: Stopped target Swaps. gitlab # [ 3.036135] systemd[1]: Stopped target Timer Units. gitlab # [ 3.036904] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.037885] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.038716] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.039752] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.040649] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 3.041520] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.042360] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.043292] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.044751] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.045756] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.046470] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.047533] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.048681] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.049670] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.050545] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.051308] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.052287] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.053263] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.054716] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.056047] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.057233] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.057992] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.058917] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.059865] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 3.060978] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 3.061895] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.062873] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.082621] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.083601] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.084656] systemd[1]: systemd-udevd.service: Consumed 2.048s CPU time over 1.354s wall clock time, 28.4M memory peak. gitlab # [ 3.086069] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.087042] systemd[1]: Closed udev Control Socket. gitlab # [ 3.087733] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.088581] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.089661] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.090681] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.091811] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.092887] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.093866] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 3.127056] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 3.128819] systemd[1]: Finished Cleanup udev Database. gitlab # [ 3.129982] systemd[1]: Reached target Switch Root. gitlab # [ 3.131058] systemd[1]: Starting NixOS Activation... gitlab # [ 3.330218] initrd-nixos-activation-start[218]: booting system configuration /nix/store/fdfalam2a4qbn0d922kdr5380mx3zxnp-nixos-system-gitlab-test gitlab # [ 3.392071] initrd-nixos-activation-start[218]: running activation script... gitlab # [ 3.708325] initrd-nixos-activation-start[243]: setting up /etc... gitlab # [ 3.881990] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 3.885355] systemd[1]: Finished NixOS Activation. gitlab # [ 3.887504] systemd[1]: Starting Switch Root... gitlab # [ 3.942880] systemd[1]: Switching root. gitlab # [ 4.063593] systemd-journald[95]: Received SIGTERM from PID 1 (systemd). gitlab # [ 4.671486] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 4.671626] systemd[1]: Detected virtualization qemu. gitlab # [ 4.671720] systemd[1]: Detected architecture arm64. gitlab # [ 4.671896] systemd[1]: Detected first boot. gitlab # [ 4.673299] systemd[1]: Initializing machine ID from random generator. gitlab # [ 4.908068] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.040531] systemd[1]: Applying preset policy. gitlab # [ 5.361642] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.559105] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.559403] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.561059] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.568847] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.572890] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.577049] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.578536] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.579970] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.581180] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.582284] systemd[1]: Expecting device /dev/ttyAMA0... gitlab # [ 5.583515] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.584683] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.585805] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.586921] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.588071] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.589247] systemd[1]: Reached target Path Units. gitlab # [ 5.590392] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.591615] systemd[1]: Reached target Slice Units. gitlab # [ 5.592781] systemd[1]: Reached target Swaps. gitlab # [ 5.596926] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.601314] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.604570] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.608319] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.609619] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.618492] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.620482] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.623923] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.625233] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.626183] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.627050] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.632075] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.633213] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.634345] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.679291] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.685068] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.690467] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.698341] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.705070] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.715400] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 5.718969] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 5.734937] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.735832] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.738948] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.753519] systemd[1]: Starting Load Kernel Module fuse... gitlab # [ 5.754164] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.762233] systemd[1]: Starting Journal Service... gitlab # [ 5.767895] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.777226] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 5.788349] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 5.789799] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.798668] fuse: init (API version 7.45) gitlab # [ 5.805273] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 5.814249] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 5.821136] systemd-journald[316]: Collecting audit messages is enabled. gitlab # [ 5.821262] systemd[1]: Mounted Huge Pages File System. gitlab # [ 5.822289] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 5.822828] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 5.823350] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 5.824294] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 5.825126] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 5.825760] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 5.826360] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.828196] systemd[1]: modprobe@fuse.service: Deactivated successfully. gitlab # [ 5.828995] systemd[1]: Finished Load Kernel Module fuse. gitlab # [ 5.833273] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.846841] EXT4-fs (vda): re-mounted d31ce65a-c197-46b2-a916-9f9d6830db7f. gitlab # [ 5.847339] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.849703] systemd[1]: Started Journal Service. gitlab # [ 5.846922] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 5.856461] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.858159] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 5.860364] systemd-modules-load[317]: Using 3 probe threads gitlab # [ 5.861901] systemd-modules-load[317]: Module 'atkbd' is built in gitlab # [ 5.863429] systemd-modules-load[317]: Module 'loop' is built in gitlab # [ 5.865003] systemd-modules-load[317]: Inserted module 'tls' gitlab # [ 5.866449] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 5.867929] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.869299] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.872232] systemd[1]: Starting Firewall... gitlab # [ 5.875321] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.878551] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.881738] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 5.886758] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 5.891077] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.895028] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.903807] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 5.909964] systemd-journald[316]: Received client request to flush runtime journal. gitlab # [ 5.947464] systemd-oomd[318]: No swap; memory pressure usage will be degraded gitlab # [ 5.950197] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 5.951371] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 5.952616] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.953587] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.954673] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.955781] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.956931] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.958080] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.959254] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.006894] systemd-udevd[353]: Using default interface naming scheme 'v261'. gitlab # [ 6.040136] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.164481] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.217669] systemd-vconsole-setup[443]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.220472] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.255744] systemd[1]: Finished Firewall. gitlab # [ 6.519847] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.562871] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.612975] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.624160] systemd[1]: Reached target Local File Systems. gitlab # [ 6.626807] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.632438] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 6.638374] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.647597] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.648619] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.656171] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.666115] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.674985] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.714960] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.716189] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.728170] (udev-worker)[420]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.740676] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.742752] (udev-worker)[419]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.747004] (udev-worker)[419]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.751233] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.755346] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.766569] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. gitlab # [ 6.795869] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.799368] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.803448] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.805305] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.806636] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.811079] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.812853] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.814193] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.815369] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.817823] systemd[1]: Starting Update is Completed... gitlab # [ 6.846644] systemd[1]: Finished Update is Completed. gitlab # [ 6.909175] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.[ 6.911942] mousedev: PS/2 mouse device common for all mice gitlab # gitlab # [ 6.914202] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 6.915457] systemd[1]: suid-sgid-wrappers.service: Consumed 147ms CPU time over 264ms wall clock time, 6.4M memory peak, 1.9M read from disk. gitlab # [ 7.121378] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.122366] systemd[1]: Reached target System Initialization. gitlab # [ 7.123207] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.124521] systemd[1]: Started logrotate.timer. gitlab # [ 7.125224] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.126337] systemd[1]: Reached target Timer Units. gitlab # [ 7.127072] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.127989] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.137473] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.138532] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.139717] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.140629] systemd[1]: Reached target Socket Units. gitlab # [ 7.143308] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.174157] dbus-broker-launch[565]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.175818] dbus-broker-launch[565]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.177248] dbus-broker-launch[565]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.179972] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.190493] dbus-broker-launch[565]: Ready gitlab # [ 7.191951] systemd[1]: Reached target Basic System. gitlab # [ 7.194850] systemd[1]: Started backdoor.service. gitlab # [ 7.216745] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.220190] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.221321] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.226100] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.237981] logrotate[570]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.241591] logrotate[570]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf gitlab # connecting to host... gitlab # [ 7.245029] logrotate[570]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.246075] logrotate[570]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.247008] logrotate[570]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.247922] logrotate[570]: Allocating hash table for state file, size 64 entries gitlab # [ 7.249159] logrotate[570]: Handling 4 logs gitlab # [ 7.249904] logrotate[570]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.252581] logrotate[570]: considering log /var/log/btmp gitlab # [ 7.253356] logrotate[570]: Creating new state gitlab # [ 7.254109] logrotate[570]: Now: 2026-09-05 11:53 gitlab # [ 7.254803] logrotate[570]: Last rotated at 2026-09-05 11:00 gitlab # [ 7.255586] logrotate[570]: log does not need rotating (log has already been rotated) gitlab # [ 7.256787] logrotate[570]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.263651] logrotate[570]: considering log /var/log/wtmp gitlab # [ 7.266744] logrotate[570]: Creating new state gitlab # [ 7.267362] logrotate[570]: Now: 2026-09-05 11:53 gitlab # [ 7.268275] logrotate[570]: Last rotated at 2026-09-05 11:00 gitlab # [ 7.269943] logrotate[570]: log does not need rotating (log has already been rotated) gitlab # [ 7.272481] logrotate[570]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.276409] logrotate[570]: switching euid from 0 to 165 and egid from 0 to 165 (pid 570) gitlab # [ 7.279451] logrotate[570]: considering log /var/gitlab/state/log/*.log gitlab: connected to guest root shell gitlab # [ 7.280464] logrotate[570]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab: (connecting took 7.74 seconds) gitlab # [ 7.282318] logrotate[570]: Creating new state gitlab: (finished: waiting for the VM to finish booting, in 7.74 seconds) gitlab # [ 7.282916] logrotate[570]: switching euid from 165 to 0 and egid from 165 to 0 (pid 570) gitlab # [ 7.283877] logrotate[570]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.287147] logrotate[570]: switching euid from 0 to 60 and egid from 0 to 60 (pid 570) gitlab # [ 7.290958] logrotate[570]: considering log /var/log/nginx/*.log gitlab # [ 7.291895] logrotate[570]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.296108] logrotate[570]: Creating new state gitlab # [ 7.296892] logrotate[570]: not running postrotate script, since no logs were rotated gitlab # [ 7.297892] logrotate[570]: switching euid from 60 to 0 and egid from 60 to 0 (pid 570) gitlab # [ 7.299424] nsncd[573]: Sep 05 11:53:54.169 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.302668] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.304854] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.306132] systemd[1]: Starting resolvconf update... gitlab # [ 7.306970] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.307854] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.309042] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.309892] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.310777] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.311638] systemd[1]: Starting User Login Management... gitlab # [ 7.366332] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.401269] systemd-logind[601]: New seat seat0. gitlab # [ 7.406842] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.408005] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.409598] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.410505] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.411788] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.412798] systemd-logind[601]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) gitlab # [ 7.413980] systemd-logind[601]: Watching system buttons on /dev/input/event0 (gpio-keys) gitlab # [ 7.417136] systemd[1]: Started User Login Management. gitlab # [ 7.417936] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.419626] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.421972] systemd[1]: nscd.service: Consumed 19ms CPU time over 191ms wall clock time, 7.2M memory peak, 5.8M read from disk. gitlab # [ 7.476139] systemd[1]: Starting linger-users.service... gitlab # [ 7.516625] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.520113] systemd[1]: Finished resolvconf update. gitlab # [ 7.522052] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 gitlab # [ 7.525726] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.527278] systemd[1]: Finished linger-users.service. gitlab # [ 7.532067] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.534306] nsncd[678]: Sep 05 11:53:54.416 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.536589] systemd[1]: Starting DHCP Client... gitlab # [ 7.538753] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.542904] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.543795] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.544977] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.546316] systemd[1]: systemd-vconsole-setup.service: Consumed 31ms CPU time over 1.378s wall clock time, 3.1M memory peak, 1.1M read from disk. gitlab # [ 7.548076] systemd[1]: Stopping Virtual Console Setup... gitlab # [ 7.548806] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.549586] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.551054] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.554743] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.555716] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.560965] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.573277] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.605730] network-addresses-eth1-start[693]: adding address 192.168.1.1/24... done gitlab # [ 7.611120] systemd-vconsole-setup[697]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 7.614898] network-addresses-eth1-start[693]: adding address 2001:db8:1::1/64... done gitlab # [ 7.616482] systemd-logind[601]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.617652] systemd[1]: Finished Virtual Console Setup. gitlab # [ 7.636754] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.657119] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.658270] systemd[1]: Reached target Network. gitlab # [ 7.663924] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.665292] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.669060] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.673003] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.679114] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.683904] dhcpcd[717]: dhcpcd-10.3.2 starting gitlab # [ 7.687376] dhcpcd[768]: dev: loaded udev gitlab # [ 7.699967] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.700367] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.719355] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.722168] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.727324] systemd[1]: Started Getty on tty1. gitlab # [ 7.733158] systemd[1]: Reached target Login Prompts. gitlab # [ 7.753966] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.764113] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.764637] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.766137] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.766442] cfg80211: failed to load regulatory.db gitlab # [ 7.804223] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.802902] dhcpcd[768]: eth0: waiting for carrier gitlab # [ 7.805033] dhcpcd[768]: eth0: carrier acquired gitlab # [ 7.809022] dhcpcd[768]: DUID 00:01:00:01:32:2e:bf:52:52:54:00:12:34:56 gitlab # [ 7.810118] dhcpcd[768]: eth0: IAID 00:12:34:56 gitlab # [ 7.810755] dhcpcd[768]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.839101] nginx-pre-start[794]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok gitlab # [ 7.842724] nginx-pre-start[794]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful gitlab # [ 7.844567] systemd[1]: Started Nginx Web Server. gitlab # [ 7.863632] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.866372] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.867491] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started gitlab # [ 7.869291] redis-server[788]: Configuration loaded gitlab # [ 7.870582] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.871447] redis[788]: monotonic clock: ARM CNTVCT @ 1000 ticks/us gitlab # [ 7.872539] redis[788]: Running mode=standalone, port=0. gitlab # [ 7.873851] redis[788]: Server initialized gitlab # [ 7.875320] postgresql-pre-start[793]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.877903] postgresql-pre-start[793]: This user must also own the server process. gitlab # [ 7.879822] redis[788]: Ready to accept connections unix gitlab # [ 7.881080] postgresql-pre-start[793]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.882539] postgresql-pre-start[793]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.884034] postgresql-pre-start[793]: The default text search configuration will be set to "english". gitlab # [ 7.885237] postgresql-pre-start[793]: Data page checksums are enabled. gitlab # [ 7.886133] postgresql-pre-start[793]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.887396] postgresql-pre-start[793]: creating subdirectories ... ok gitlab # [ 7.888266] postgresql-pre-start[793]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.889410] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.891447] dhcpcd[768]: eth0: soliciting a DHCP lease gitlab # [ 7.896525] dhcpcd[768]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 7.912260] dhcpcd[768]: eth0: probing address 10.0.2.15/24 gitlab # [ 7.914671] postgresql-pre-start[793]: selecting default "max_connections" ... 100 gitlab # [ 7.972640] postgresql-pre-start[793]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.277739] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.471504] postgresql-pre-start[793]: selecting default time zone ... UTC gitlab # [ 8.475255] postgresql-pre-start[793]: creating configuration files ... ok gitlab # [ 8.495157] gitlab-workhorse[823]: time="2026-09-05T11:53:55Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.500692] gitlab-workhorse[823]: time="2026-09-05T11:53:55Z" level=info msg="Using redis/go-redis" gitlab # [ 8.504585] gitlab-workhorse[823]: time="2026-09-05T11:53:55Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.709064] postgresql-pre-start[793]: running bootstrap script ... ok gitlab # [ 9.401828] postgresql-pre-start[793]: performing post-bootstrap initialization ... ok gitlab # [ 9.748277] postgresql-pre-start[793]: syncing data to disk ... ok gitlab # [ 9.751003] postgresql-pre-start[793]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.755024] postgresql-pre-start[793]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.761510] postgresql-pre-start[793]: Success. You can now start the database server using: gitlab # [ 9.764945] postgresql-pre-start[793]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.841130] dhcpcd[768]: eth0: soliciting an IPv6 router gitlab # [ 9.843710] dhcpcd[768]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.846436] dhcpcd[768]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.850243] dhcpcd[768]: eth0: adding route to fec0::/64 gitlab # [ 9.852650] dhcpcd[768]: eth0: adding default route via fe80::2 gitlab # [ 9.943048] postgres[858]: [858] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.948890] postgres[858]: [858] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.952520] postgres[858]: [858] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.956884] postgres[858]: [858] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.962724] postgres[872]: [872] LOG: database system was shut down at 2026-09-05 11:53:56 GMT gitlab # [ 9.968206] postgres[858]: [858] LOG: database system is ready to accept connections gitlab # [ 9.972312] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.974453] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.035010] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[865]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 10.039253] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[865]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 10.127558] systemd[1]: Finished gitlab-config.service. gitlab # [ 10.132580] systemd[1]: Started gitaly.service. gitlab # [ 10.342351] postgresql-setup-start[890]: CREATE DATABASE gitlab # [ 10.404126] postgresql-setup-start[902]: CREATE ROLE gitlab # [ 10.432676] postgresql-setup-start[909]: ALTER DATABASE gitlab # [ 10.498395] postgresql-setup-start[914]: CREATE ROLE gitlab # [ 10.505062] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 10.507558] systemd[1]: Reached target PostgreSQL. gitlab # [ 10.512270] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 10.517811] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 10.710471] gitlab-postgresql-start[933]: CREATE DATABASE gitlab # [ 10.777454] gitlab-postgresql-start[940]: CREATE EXTENSION gitlab # [ 10.886877] gitlab-postgresql-start[942]: CREATE EXTENSION gitlab # [ 10.897834] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.902368] gitaly[897]: time="2026-09-05T11:53:57.784Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=897 gitlab # [ 10.911440] gitaly[897]: time="2026-09-05T11:53:57.784Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=897 gitlab # [ 10.915216] gitaly[897]: time="2026-09-05T11:53:57.784Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=897 gitlab # [ 10.920120] gitaly[897]: time="2026-09-05T11:53:57.784Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=897 gitlab # [ 10.924131] gitaly[897]: time="2026-09-05T11:53:57.784Z" level=info msg="Starting Gitaly" pid=897 version= gitlab # [ 11.121315] gitaly[897]: time="2026-09-05T11:53:58.003Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=218 pid=897 gitlab # [ 11.123347] gitaly[897]: time="2026-09-05T11:53:58.004Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=897 gitlab # [ 11.125168] gitaly[897]: time="2026-09-05T11:53:58.004Z" level=info msg="finished initializing command factory" duration_ms=0 pid=897 gitlab # [ 11.126811] gitaly[897]: time="2026-09-05T11:53:58.004Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-897/git-exec-1237131856.d/git pid=897 gitlab # [ 11.128702] gitaly[897]: time="2026-09-05T11:53:58.008Z" level=info msg="finished detecting git version" duration_ms=4 pid=897 gitlab # [ 11.130173] gitaly[897]: time="2026-09-05T11:53:58.009Z" level=info msg="using Git version" pid=897 version=2.53.g gitlab # [ 11.131612] gitaly[897]: time="2026-09-05T11:53:58.011Z" level=info msg="clearing disk cache object folder" pid=897 storage=default gitlab # [ 11.133262] gitaly[897]: time="2026-09-05T11:53:58.011Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache922267467 pid=897 storage=default gitlab # [ 11.135793] gitaly[897]: time="2026-09-05T11:53:58.011Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache922267467 pid=897 storage=default gitlab # [ 11.139202] gitaly[897]: time="2026-09-05T11:53:58.011Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=897 gitlab # [ 11.142376] gitaly[897]: time="2026-09-05T11:53:58.011Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=897 gitlab # [ 11.145012] gitaly[897]: time="2026-09-05T11:53:58.011Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache922267467 clear_duration_ms=0 pid=897 storage=default gitlab # [ 11.148035] gitaly[897]: time="2026-09-05T11:53:58.026Z" level=info msg="counting repositories" pid=897 gitlab # [ 11.149271] gitaly[897]: time="2026-09-05T11:53:58.026Z" level=info msg="starting to count repositories" pid=897 storage_path=/var/gitlab/state/repositories gitlab # [ 11.151350] gitaly[897]: time="2026-09-05T11:53:58.026Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=897 storage_path=/var/gitlab/state/repositories gitlab # [ 11.153805] gitaly[897]: time="2026-09-05T11:53:58.026Z" level=info msg="completed counting all repositories" counting_duration=0 pid=897 gitlab # [ 11.155424] gitaly[897]: time="2026-09-05T11:53:58.026Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=897 gitlab # [ 11.157705] gitaly[897]: time="2026-09-05T11:53:58.026Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=897 schema=unix gitlab # [ 11.160033] gitaly[897]: time="2026-09-05T11:53:58.026Z" level=info msg="listening at address" address=/run/gitaly/gitaly-897/sock.d/intern pid=897 schema=unix gitlab # [ 11.162142] gitaly[897]: time="2026-09-05T11:53:58.026Z" level=info msg="finished tempdir cleaner walk" pid=897 storage=default time_ms=0 gitlab # [ 11.163825] gitaly[897]: time="2026-09-05T11:53:58.027Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=897 gitlab # [ 11.165455] gitaly[897]: time="2026-09-05T11:53:58.027Z" level=info msg="Started Gitaly" duration_ms=243 pid=897 gitlab # [ 11.166693] gitaly[897]: time="2026-09-05T11:53:58.028Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=897 gitlab # [ 11.168183] gitaly[897]: time="2026-09-05T11:53:58.028Z" level=info msg="maintenance: daily scheduled" pid=897 scheduled="2026-09-05 12:00:00 +0000 UTC" gitlab # [ 11.782074] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[924]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.782977] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[924]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.818011] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[924]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.236258] dhcpcd[768]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.236558] dhcpcd[768]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.236692] dhcpcd[768]: eth0: adding default route via 10.0.2.2 gitlab # [ 13.445414] systemd[1]: Started DHCP Client. gitlab # [ 13.450088] systemd[1]: Reached target Network is Online. gitlab # [ 14.657014] gitaly[897]: time="2026-09-05T11:54:01.539Z" level=info msg="License database preloaded" duration_ms=3754 pid=897 gitlab # [ 58.123664] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[924]: W, [2026-09-05T11:54:45.005509 #924] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 58.551305] gitlab-workhorse[823]: time="2026-09-05T11:54:45Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 61.261029] postgres[1450]: [1450] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 61.265660] postgres[1450]: [1450] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 61.269475] postgres[1450]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 61.273506] postgres[1450]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 61.277279] postgres[1450]: attidentity AS identity, gitlab # [ 61.279827] postgres[1450]: attgenerated as attgenerated gitlab # [ 61.283656] postgres[1450]: FROM pg_attribute a gitlab # [ 61.285914] postgres[1450]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 61.289735] postgres[1450]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 61.292530] postgres[1450]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 61.296722] postgres[1450]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 61.299615] postgres[1450]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 61.302418] postgres[1450]: ORDER BY a.attnum gitlab # [ 61.304537] postgres[1450]: gitlab # [ 67.499082] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1510]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 67.499495] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1510]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 68.559039] gitlab-workhorse[823]: time="2026-09-05T11:54:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 78.566332] gitlab-workhorse[823]: time="2026-09-05T11:55:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 88.574650] gitlab-workhorse[823]: time="2026-09-05T11:55:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 92.472884] postgres[1604]: [1604] ERROR: relation "table_without_models" does not exist gitlab # [ 92.473380] postgres[1604]: [1604] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 98.583778] gitlab-workhorse[823]: time="2026-09-05T11:55:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 108.592568] gitlab-workhorse[823]: time="2026-09-05T11:55:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.593844] gitlab-workhorse[823]: time="2026-09-05T11:55:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 123.809541] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 123.813844] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 123.817279] systemd[1]: Starting gitlab.service... gitlab # [ 124.416244] bundle[1988]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 124.422455] bundle[1988]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 124.428459] bundle[1988]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 128.597693] gitlab-workhorse[823]: time="2026-09-05T11:55:55Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 129.162054] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.014Z","pid":1988,"message":"* Enabling systemd notification integration"} gitlab # [ 129.167062] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"Puma starting in cluster mode..."} gitlab # [ 129.171512] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 129.176748] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 129.182764] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Min threads: 0"} gitlab # [ 129.186724] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Max threads: 4"} gitlab # [ 129.190756] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Environment: production"} gitlab # [ 129.195087] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Master PID: 1988"} gitlab # [ 129.199164] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Workers: 2"} gitlab # [ 129.203103] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 129.208197] bundle[1988]: {"timestamp":"2026-09-05T11:55:56.044Z","pid":1988,"message":"* Preloading application"} gitlab # [ 129.213109] sidekiq-cluster[1987]: {"severity":"INFO","time":"2026-09-05T11:55:56.027Z","message":"Starting cluster with 1 processes"} gitlab # [ 138.599129] gitlab-workhorse[823]: time="2026-09-05T11:56:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 148.611548] gitlab-workhorse[823]: time="2026-09-05T11:56:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 158.613009] gitlab-workhorse[823]: time="2026-09-05T11:56:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 168.624115] gitlab-workhorse[823]: time="2026-09-05T11:56:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 168.751834] bundle[1988]: W, [2026-09-05T11:56:35.634142 #1988] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 172.445891] sidekiq-cluster[2037]: 2026-09-05T11:56:39.328Z pid=2037 tid=wnx INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2037, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431198208} gitlab # [ 172.742275] sidekiq-cluster[2037]: W, [2026-09-05T11:56:39.625559 #2037] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 172.809948] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:39.693Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 176.756120] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.638Z","message":"GitLab reliable fetch activated!"} gitlab # [ 176.756601] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.638Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 176.789847] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.673Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 176.796901] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.679Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 176.803932] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.686Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 176.810597] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.692Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 176.817968] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.698Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 176.824340] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.704Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 176.830403] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.710Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 176.835421] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.714Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 176.840282] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.717Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 176.845335] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.720Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 176.850080] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.722Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 176.854650] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.725Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 176.858890] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.728Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 176.863428] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.730Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 176.867903] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.732Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 176.872442] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.735Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 176.876334] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.738Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 176.879865] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.742Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 176.883814] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.745Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 176.887438] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.748Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 176.891217] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.751Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 176.894975] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.753Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 176.898114] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.756Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 176.901217] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.759Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 176.904193] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.761Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 176.906898] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.764Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 176.909584] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.767Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 176.912129] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.770Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 176.914659] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.773Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 176.917385] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.776Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 176.919657] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.778Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 176.922387] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.782Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 176.924714] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.784Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 176.927064] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.787Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 176.929436] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.789Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 176.931648] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.792Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 176.934038] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.794Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 176.936196] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.797Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 176.938322] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.799Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 176.940428] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.800Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 176.942548] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.803Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 176.944686] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.804Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 176.946815] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.806Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 176.948922] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.808Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 176.951015] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.811Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 176.953277] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.813Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 176.955324] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.815Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 176.957493] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.817Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 176.959488] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.819Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 176.961736] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.822Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 176.963794] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.824Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 176.966007] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.826Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 176.968068] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.838Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 176.970356] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.841Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 176.972541] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.843Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 176.974653] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.846Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 176.976824] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.848Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 176.979057] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.851Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 176.981360] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.853Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 176.983553] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.855Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 176.985702] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.857Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 176.987709] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.859Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 176.989968] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.862Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 176.992093] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.864Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 176.994278] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.867Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 176.996460] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.869Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 176.998505] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.871Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 177.000678] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.873Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 177.002839] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.875Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 177.005131] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.879Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 177.007306] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.883Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 177.009552] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.884Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 177.011722] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.886Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 177.014209] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.888Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 177.016550] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.890Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 177.018878] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.892Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 177.021147] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.894Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 177.023539] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.896Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 177.025834] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.898Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 177.027923] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.899Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 177.030437] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.901Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 177.032910] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.903Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 177.035171] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.906Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 177.037477] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.908Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 177.039782] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.910Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 177.042132] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.913Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 177.044504] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.918Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 177.047697] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.922Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 177.050127] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.924Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 177.052401] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.926Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 177.054659] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.928Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 177.056972] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.930Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 177.059211] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.932Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 177.061697] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.934Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 177.063601] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.936Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 177.066025] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.939Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 177.068212] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.941Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 177.070372] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.944Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 177.073627] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.957Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 177.076158] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.959Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 177.078358] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.961Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 177.081807] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.965Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 177.088814] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.972Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 177.094670] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:43.978Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 177.156265] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:44.037Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 177.161910] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:44.042Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 177.166986] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:44.044Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 177.172477] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:44.047Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 177.177720] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:56:44.051Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 178.634204] gitlab-workhorse[823]: time="2026-09-05T11:56:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 188.635401] gitlab-workhorse[823]: time="2026-09-05T11:56:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 198.636499] gitlab-workhorse[823]: time="2026-09-05T11:57:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 208.637990] gitlab-workhorse[823]: time="2026-09-05T11:57:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 216.763016] postgres[2775]: [2775] ERROR: relation "table_without_models" does not exist gitlab # [ 216.763541] postgres[2775]: [2775] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 218.639093] gitlab-workhorse[823]: time="2026-09-05T11:57:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 219.404352] postgres[2795]: [2795] ERROR: relation "table_without_models" does not exist gitlab # [ 219.407759] postgres[2795]: [2795] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 221.718139] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:28.601Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 221.721390] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:28.604Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 221.723358] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:28.604Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 221.725183] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:28.604Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 221.732070] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:28.613Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 224.410296] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:31.293Z","message":"Cleaning working queues"} gitlab # [ 224.414751] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:31.297Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"47958e8c0c395f5c7340ad57","created_at":"2026-09-05T11:55:50.028Z","trace_propagation_headers":{"sentry-trace":"0531bf1258ed4e78910e74e0ec2cec63-bd40ecb5f94c4579","baggage":"sentry-trace_id=0531bf1258ed4e78910e74e0ec2cec63,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6ecdd4eac4601805381d71d474fb3c90","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-05T11:55:50.039Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-47958e8c0c395f5c7340ad57: start","job_status":"start","queue_duration_s":101.257347,"scheduling_latency_s":101.25745} gitlab # [ 224.793688] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:31.676Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"47958e8c0c395f5c7340ad57","created_at":"2026-09-05T11:55:50.028Z","trace_propagation_headers":{"sentry-trace":"0531bf1258ed4e78910e74e0ec2cec63-bd40ecb5f94c4579","baggage":"sentry-trace_id=0531bf1258ed4e78910e74e0ec2cec63,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6ecdd4eac4601805381d71d474fb3c90","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-05T11:55:50.039Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-47958e8c0c395f5c7340ad57: done: 0.379136 sec","job_status":"done","queue_duration_s":101.257347,"scheduling_latency_s":101.25745,"redis_calls":29,"redis_duration_s":0.011889,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.006943,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003111,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001835,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.197327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.20298428000000002,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.379136,"completed_at":"2026-09-05T11:57:31.676Z","load_balancing_strategy":"primary","db_duration_s":0.102667,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 226.237950] bundle[1988]: {"timestamp":"2026-09-05T11:57:33.120Z","pid":1988,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 226.238254] bundle[1988]: {"timestamp":"2026-09-05T11:57:33.120Z","pid":1988,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 226.238427] bundle[1988]: {"timestamp":"2026-09-05T11:57:33.120Z","pid":1988,"message":"! #\u003cThread:0x0000f16087474440 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 226.238575] bundle[1988]: {"timestamp":"2026-09-05T11:57:33.120Z","pid":1988,"message":"! #\u003cThread:0x0000f16086ce4478 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 226.238690] bundle[1988]: {"timestamp":"2026-09-05T11:57:33.120Z","pid":1988,"message":"Use Ctrl-C to stop"} gitlab # [ 226.238798] bundle[1988]: {"timestamp":"2026-09-05T11:57:33.121Z","pid":1988,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 226.965557] bundle[1988]: {"timestamp":"2026-09-05T11:57:33.848Z","pid":1988,"message":"- Worker 0 (PID: 2912) booted in 0.67s, phase: 0"} gitlab # [ 227.023518] bundle[1988]: {"timestamp":"2026-09-05T11:57:33.906Z","pid":1988,"message":"- Worker 1 (PID: 2916) booted in 0.69s, phase: 0"} gitlab # [ 227.029462] systemd[1]: Started gitlab.service. gitlab # [ 227.031685] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 227.034865] systemd[1]: Starting setup-gitlab.service... gitlab # [ 227.076575] setup-gitlab-start[2949]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 230.711067] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:57:37 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 3590 gitlab # [ 230.717024] setup-gitlab-start[2949]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 230.718849] setup-gitlab-start[2949]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 230.735224] runuser[2980]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 231.301572] setup-gitlab-start[2981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.308077] setup-gitlab-start[2981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.314263] setup-gitlab-start[2981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 232.196975] setup-gitlab-start[2981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 232.197308] setup-gitlab-start[2981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 232.199579] setup-gitlab-start[2981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 239.077746] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:45.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"51ad0d27a993089207a9445e","created_at":"2026-09-05T11:57:45.784Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db71c505e26d513d764553f7812244f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:45.941Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-51ad0d27a993089207a9445e: start","job_status":"start","queue_duration_s":0.012672,"scheduling_latency_s":0.012787} gitlab # [ 239.690977] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:46.571Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"51ad0d27a993089207a9445e","created_at":"2026-09-05T11:57:45.784Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db71c505e26d513d764553f7812244f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:45.941Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-51ad0d27a993089207a9445e: done: 0.617186 sec","job_status":"done","queue_duration_s":0.012672,"scheduling_latency_s":0.012787,"redis_calls":12,"redis_duration_s":0.052413,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.008581,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.027939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.015893,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.139285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0050275,"duration_s":0.617186,"completed_at":"2026-09-05T11:57:46.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.091719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.793115] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:46.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"91640e7402381b73f0c0acf3","created_at":"2026-09-05T11:57:46.059Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20fe55c1ff438fda938c2f87b239f33","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.086Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-91640e7402381b73f0c0acf3: start","job_status":"start","queue_duration_s":0.501988,"scheduling_latency_s":0.502121} gitlab # [ 239.948249] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:46.830Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 240.076588] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:46.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"91640e7402381b73f0c0acf3","created_at":"2026-09-05T11:57:46.059Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20fe55c1ff438fda938c2f87b239f33","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.086Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-91640e7402381b73f0c0acf3: done: 0.365511 sec","job_status":"done","queue_duration_s":0.501988,"scheduling_latency_s":0.502121,"redis_calls":10,"redis_duration_s":0.057006,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.011456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.03298,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.01257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.115697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00464384,"duration_s":0.365511,"completed_at":"2026-09-05T11:57:46.953Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008302,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.181008] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:46.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2fe96b4cbae51029887b3053","created_at":"2026-09-05T11:57:46.101Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f89f0c48b2fd1ece496c7df19beb29d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.165Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2fe96b4cbae51029887b3053: start","job_status":"start","queue_duration_s":0.80251,"scheduling_latency_s":0.80257} gitlab # [ 240.212669] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:47.086Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 240.229288] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:47.111Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2fe96b4cbae51029887b3053","created_at":"2026-09-05T11:57:46.101Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f89f0c48b2fd1ece496c7df19beb29d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.165Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2fe96b4cbae51029887b3053: done: 0.143925 sec","job_status":"done","queue_duration_s":0.80251,"scheduling_latency_s":0.80257,"redis_calls":10,"redis_duration_s":0.006765,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002925,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.071,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.071,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00061714,"duration_s":0.143925,"completed_at":"2026-09-05T11:57:47.111Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.070496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.297233] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:47.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7f2626dbc0ee8506ddf514bb","created_at":"2026-09-05T11:57:46.254Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e29f6e14f00883c8f73607a860de88d9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.273Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7f2626dbc0ee8506ddf514bb: start","job_status":"start","queue_duration_s":0.842922,"scheduling_latency_s":0.842963} gitlab # [ 240.672715] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:47.554Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7f2626dbc0ee8506ddf514bb","created_at":"2026-09-05T11:57:46.254Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e29f6e14f00883c8f73607a860de88d9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.273Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7f2626dbc0ee8506ddf514bb: done: 0.438894 sec","job_status":"done","queue_duration_s":0.842922,"scheduling_latency_s":0.842963,"redis_calls":12,"redis_duration_s":0.006866000000000001,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002125,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002425,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.134731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00195034,"exclusive_lock_hold_duration_s":0.21589006000000002,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.438894,"completed_at":"2026-09-05T11:57:47.554Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.104448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.757407] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:47.558Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"720c14de816c87dd08424259","created_at":"2026-09-05T11:57:46.301Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d674be224f1000a914b7854f9cd188e2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.321Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-720c14de816c87dd08424259: start","job_status":"start","queue_duration_s":1.237535,"scheduling_latency_s":1.237582} gitlab # [ 240.773107] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:47.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"720c14de816c87dd08424259","created_at":"2026-09-05T11:57:46.301Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d674be224f1000a914b7854f9cd188e2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.321Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-720c14de816c87dd08424259: done: 0.049412 sec","job_status":"done","queue_duration_s":1.237535,"scheduling_latency_s":1.237582,"redis_calls":10,"redis_duration_s":0.0036910000000000003,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000648,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002053,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049144,"duration_s":0.049412,"completed_at":"2026-09-05T11:57:47.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.817586] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:47.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a851c1ea0da586227351ed64","created_at":"2026-09-05T11:57:46.385Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2797ae1a385c3cd5b4dca62c9340fc4b","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.450Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a851c1ea0da586227351ed64: start","job_status":"start","queue_duration_s":1.159412,"scheduling_latency_s":1.159437} gitlab # [ 241.154745] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a851c1ea0da586227351ed64","created_at":"2026-09-05T11:57:46.385Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2797ae1a385c3cd5b4dca62c9340fc4b","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.450Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a851c1ea0da586227351ed64: done: 0.426481 sec","job_status":"done","queue_duration_s":1.159412,"scheduling_latency_s":1.159437,"redis_calls":565,"redis_duration_s":0.105395,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.103511,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.363406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002955,"duration_s":0.426481,"completed_at":"2026-09-05T11:57:48.036Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.265173] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.038Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"20e7759e8198bf38ecade86b","created_at":"2026-09-05T11:57:46.550Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bf8b49fb3700e37d4511dad09a845ae","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.607Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-20e7759e8198bf38ecade86b: start","job_status":"start","queue_duration_s":1.430334,"scheduling_latency_s":1.430354} gitlab # [ 241.287662] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"20e7759e8198bf38ecade86b","created_at":"2026-09-05T11:57:46.550Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bf8b49fb3700e37d4511dad09a845ae","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.607Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-20e7759e8198bf38ecade86b: done: 0.024838 sec","job_status":"done","queue_duration_s":1.430334,"scheduling_latency_s":1.430354,"redis_calls":12,"redis_duration_s":0.0028580000000000003,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001304,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001097,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005316,"exclusive_lock_hold_duration_s":0.0043544199999999995,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.024838,"completed_at":"2026-09-05T11:57:48.062Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002805,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.344310] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.064Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"807da8ebbbe19de863dc9a5b","created_at":"2026-09-05T11:57:46.666Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b52370d866ddebbf855542134d53fcf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.690Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-807da8ebbbe19de863dc9a5b: start","job_status":"start","queue_duration_s":1.373809,"scheduling_latency_s":1.373826} gitlab # [ 241.359013] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.102Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"807da8ebbbe19de863dc9a5b","created_at":"2026-09-05T11:57:46.666Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b52370d866ddebbf855542134d53fcf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.690Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-807da8ebbbe19de863dc9a5b: done: 0.03741 sec","job_status":"done","queue_duration_s":1.373809,"scheduling_latency_s":1.373826,"redis_calls":16,"redis_duration_s":0.004239,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00235,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001407,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003167,"duration_s":0.03741,"completed_at":"2026-09-05T11:57:48.101Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.401734] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.103Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"efade877753cca950480696c","created_at":"2026-09-05T11:57:46.782Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1e6825cd6554f486ba400ce82d9ce1f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.869Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-efade877753cca950480696c: start","job_status":"start","queue_duration_s":1.234143,"scheduling_latency_s":1.234176} gitlab # [ 241.415203] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"efade877753cca950480696c","created_at":"2026-09-05T11:57:46.782Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1e6825cd6554f486ba400ce82d9ce1f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.869Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-efade877753cca950480696c: done: 0.033987 sec","job_status":"done","queue_duration_s":1.234143,"scheduling_latency_s":1.234176,"redis_calls":12,"redis_duration_s":0.005253,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047234,"exclusive_lock_hold_duration_s":0.00826002,"duration_s":0.033987,"completed_at":"2026-09-05T11:57:48.137Z","load_balancing_strategy":"primary","db_duration_s":0.003702,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.457536] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.139Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e5f398c47c522e0b2eadd381","created_at":"2026-09-05T11:57:46.935Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f31c5d509e0cb19fd83236cf01ac6693","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.988Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e5f398c47c522e0b2eadd381: start","job_status":"start","queue_duration_s":1.151138,"scheduling_latency_s":1.151158} gitlab # [ 241.471676] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.177Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e5f398c47c522e0b2eadd381","created_at":"2026-09-05T11:57:46.935Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f31c5d509e0cb19fd83236cf01ac6693","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:46.988Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e5f398c47c522e0b2eadd381: done: 0.038431 sec","job_status":"done","queue_duration_s":1.151138,"scheduling_latency_s":1.151158,"redis_calls":13,"redis_duration_s":0.0057669999999999996,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003075,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001603,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0096921,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.038431,"completed_at":"2026-09-05T11:57:48.177Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003859,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 241.515543] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.179Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"71eb4373dcef6a2b4f8f8351","created_at":"2026-09-05T11:57:47.083Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f540e2f5ff5eb694b369d6c59c5259f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.107Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-71eb4373dcef6a2b4f8f8351: start","job_status":"start","queue_duration_s":1.071577,"scheduling_latency_s":1.071593} gitlab # [ 241.529757] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.191Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 241.532888] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"71eb4373dcef6a2b4f8f8351","created_at":"2026-09-05T11:57:47.083Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f540e2f5ff5eb694b369d6c59c5259f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.107Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-71eb4373dcef6a2b4f8f8351: done: 0.019689 sec","job_status":"done","queue_duration_s":1.071577,"scheduling_latency_s":1.071593,"redis_calls":10,"redis_duration_s":0.002361,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000543,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001338,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00048,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031492,"duration_s":0.019689,"completed_at":"2026-09-05T11:57:48.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000886,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.574461] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.200Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b112d455ef4f0ad301ce4fed","created_at":"2026-09-05T11:57:47.182Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46f9274708c87625214ce1ba36d612dc","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.189Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b112d455ef4f0ad301ce4fed: start","job_status":"start","queue_duration_s":1.010981,"scheduling_latency_s":1.010997} gitlab # [ 241.588221] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.251Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b112d455ef4f0ad301ce4fed","created_at":"2026-09-05T11:57:47.182Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46f9274708c87625214ce1ba36d612dc","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.189Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b112d455ef4f0ad301ce4fed: done: 0.050453 sec","job_status":"done","queue_duration_s":1.010981,"scheduling_latency_s":1.010997,"redis_calls":10,"redis_duration_s":0.002734,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000613,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001594,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045496,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.050453,"completed_at":"2026-09-05T11:57:48.250Z","load_balancing_strategy":"primary","db_duration_s":0.026884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.631047] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"3d0fd8c468f9d2a036b6a074","created_at":"2026-09-05T11:57:47.200Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.209Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-3d0fd8c468f9d2a036b6a074: start","job_status":"start","queue_duration_s":1.043186,"scheduling_latency_s":1.043203} gitlab # [ 241.644462] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"3d0fd8c468f9d2a036b6a074","created_at":"2026-09-05T11:57:47.200Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.209Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-3d0fd8c468f9d2a036b6a074: done: 0.040825 sec","job_status":"done","queue_duration_s":1.043186,"scheduling_latency_s":1.043203,"redis_calls":19,"redis_duration_s":0.004938,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001083,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000612,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002444,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000799,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049884,"duration_s":0.040825,"completed_at":"2026-09-05T11:57:48.293Z","load_balancing_strategy":"primary","db_duration_s":0.003465,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.686006] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8d0480f3140aa608671d8bc1","created_at":"2026-09-05T11:57:47.238Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4577d1a93dc76de3349351eafc96137e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.266Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8d0480f3140aa608671d8bc1: start","job_status":"start","queue_duration_s":1.029011,"scheduling_latency_s":1.029027} gitlab # [ 241.700260] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8d0480f3140aa608671d8bc1","created_at":"2026-09-05T11:57:47.238Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4577d1a93dc76de3349351eafc96137e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.266Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8d0480f3140aa608671d8bc1: done: 0.063813 sec","job_status":"done","queue_duration_s":1.029011,"scheduling_latency_s":1.029027,"redis_calls":14,"redis_duration_s":0.0040680000000000004,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001329,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058886,"exclusive_lock_hold_duration_s":0.03849018,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.063813,"completed_at":"2026-09-05T11:57:48.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027994,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.744435] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.360Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c496b02516c90de401025ec3","created_at":"2026-09-05T11:57:47.286Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a19899e5df4ba5491300f7cca673367","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-05T11:57:47.301Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c496b02516c90de401025ec3: start","job_status":"start","queue_duration_s":1.059482,"scheduling_latency_s":1.059499} gitlab # [ 241.757287] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.398Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c496b02516c90de401025ec3","created_at":"2026-09-05T11:57:47.286Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a19899e5df4ba5491300f7cca673367","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-05T11:57:47.301Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c496b02516c90de401025ec3: done: 0.037756 sec","job_status":"done","queue_duration_s":1.059482,"scheduling_latency_s":1.059499,"redis_calls":10,"redis_duration_s":0.0026709999999999998,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000528,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001614,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000529,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034334,"duration_s":0.037756,"completed_at":"2026-09-05T11:57:48.398Z","load_balancing_strategy":"primary","db_duration_s":0.006906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.797768] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"046b08477f4785c07b150a7a","created_at":"2026-09-05T11:57:47.321Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd367c58b0c5f1f8f768bedcd284fb4c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-05T11:57:47.376Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-046b08477f4785c07b150a7a: start","job_status":"start","queue_duration_s":1.023445,"scheduling_latency_s":1.023464} gitlab # [ 241.810789] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.412Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 241.814503] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"046b08477f4785c07b150a7a","created_at":"2026-09-05T11:57:47.321Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd367c58b0c5f1f8f768bedcd284fb4c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-05T11:57:47.376Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-046b08477f4785c07b150a7a: done: 0.020608 sec","job_status":"done","queue_duration_s":1.023445,"scheduling_latency_s":1.023464,"redis_calls":10,"redis_duration_s":0.0023899999999999998,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001437,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034582,"duration_s":0.020608,"completed_at":"2026-09-05T11:57:48.420Z","load_balancing_strategy":"primary","db_duration_s":0.000652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.856135] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"901be000ff61120585a43e16","created_at":"2026-09-05T11:57:47.388Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3751183dc8d241a2f3dc6720609fadcb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.401Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-901be000ff61120585a43e16: start","job_status":"start","queue_duration_s":1.020896,"scheduling_latency_s":1.02091} gitlab # [ 241.870541] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"901be000ff61120585a43e16","created_at":"2026-09-05T11:57:47.388Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3751183dc8d241a2f3dc6720609fadcb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.401Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-901be000ff61120585a43e16: done: 0.020527 sec","job_status":"done","queue_duration_s":1.020896,"scheduling_latency_s":1.02091,"redis_calls":10,"redis_duration_s":0.002856,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000487,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032404,"duration_s":0.020527,"completed_at":"2026-09-05T11:57:48.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.912382] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e3402f56e5593ea449fcfcb4","created_at":"2026-09-05T11:57:47.413Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.423Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e3402f56e5593ea449fcfcb4: start","job_status":"start","queue_duration_s":1.020867,"scheduling_latency_s":1.020883} gitlab # [ 241.925535] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e3402f56e5593ea449fcfcb4","created_at":"2026-09-05T11:57:47.413Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.423Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e3402f56e5593ea449fcfcb4: done: 0.110845 sec","job_status":"done","queue_duration_s":1.020867,"scheduling_latency_s":1.020883,"redis_calls":71,"redis_duration_s":0.017839999999999998,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002526,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000932,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013687,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000695,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032556,"duration_s":0.110845,"completed_at":"2026-09-05T11:57:48.554Z","load_balancing_strategy":"primary","db_duration_s":0.002749,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.968177] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.556Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e6605d663f246e82d950f63a","created_at":"2026-09-05T11:57:47.430Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bc3b18e9c08c8c66edfdcab41fa45ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.439Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e6605d663f246e82d950f63a: start","job_status":"start","queue_duration_s":1.117495,"scheduling_latency_s":1.11751} gitlab # [ 241.982398] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e6605d663f246e82d950f63a","created_at":"2026-09-05T11:57:47.430Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bc3b18e9c08c8c66edfdcab41fa45ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.439Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e6605d663f246e82d950f63a: done: 0.250563 sec","job_status":"done","queue_duration_s":1.117495,"scheduling_latency_s":1.11751,"redis_calls":12,"redis_duration_s":0.002967,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000501,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001389,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001077,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.048,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.183741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053964,"exclusive_lock_hold_duration_s":0.23061394,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.250563,"completed_at":"2026-09-05T11:57:48.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.073439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.026882] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.808Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"54216fe8d7ca55535c1bd485","created_at":"2026-09-05T11:57:47.444Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea3ca03f8efbc9075e3e98c3609cbfa6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.452Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-54216fe8d7ca55535c1bd485: start","job_status":"start","queue_duration_s":1.35614,"scheduling_latency_s":1.356155} gitlab # [ 242.042742] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.821Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 242.046269] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.829Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"54216fe8d7ca55535c1bd485","created_at":"2026-09-05T11:57:47.444Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea3ca03f8efbc9075e3e98c3609cbfa6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-05T11:57:47.452Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-54216fe8d7ca55535c1bd485: done: 0.020344 sec","job_status":"done","queue_duration_s":1.35614,"scheduling_latency_s":1.356155,"redis_calls":10,"redis_duration_s":0.0024980000000000002,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000535,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001471,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000492,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031686,"duration_s":0.020344,"completed_at":"2026-09-05T11:57:48.829Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.088353] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f9e1799a1053588b088e18b7","created_at":"2026-09-05T11:57:47.472Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a09504fe63bc4de8a61e3772a15f86a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-05T11:57:47.487Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f9e1799a1053588b088e18b7: start","job_status":"start","queue_duration_s":1.343043,"scheduling_latency_s":1.343059} gitlab # [ 242.101615] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.842Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 242.104914] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f9e1799a1053588b088e18b7","created_at":"2026-09-05T11:57:47.472Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a09504fe63bc4de8a61e3772a15f86a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-05T11:57:47.487Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f9e1799a1053588b088e18b7: done: 0.018692 sec","job_status":"done","queue_duration_s":1.343043,"scheduling_latency_s":1.343059,"redis_calls":10,"redis_duration_s":0.002421,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000488,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000311,"duration_s":0.018692,"completed_at":"2026-09-05T11:57:48.849Z","load_balancing_strategy":"primary","db_duration_s":0.00064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.145677] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.851Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"24cc226d9466b93a5f52de76","created_at":"2026-09-05T11:57:48.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-6bd19fc562f1436c","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-24cc226d9466b93a5f52de76: start","job_status":"start","queue_duration_s":0.566057,"scheduling_latency_s":0.566072} gitlab # [ 242.158364] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.869Z","message":"No existing merge request to be cleaned up."} gitlab # [ 242.160044] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.884Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"24cc226d9466b93a5f52de76","created_at":"2026-09-05T11:57:48.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-6bd19fc562f1436c","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-24cc226d9466b93a5f52de76: done: 0.033334 sec","job_status":"done","queue_duration_s":0.566057,"scheduling_latency_s":0.566072,"redis_calls":13,"redis_duration_s":0.004136,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039368,"duration_s":0.033334,"completed_at":"2026-09-05T11:57:48.884Z","load_balancing_strategy":"primary","db_duration_s":0.005565,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.199674] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.886Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"978c7a5fa9052b18a7bb72f9","created_at":"2026-09-05T11:57:48.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-6bd19fc562f1436c","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-978c7a5fa9052b18a7bb72f9: start","job_status":"start","queue_duration_s":0.600927,"scheduling_latency_s":0.600943} gitlab # [ 242.211977] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.897Z","message":"No existing merge request to be cleaned up."} gitlab # [ 242.213704] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.906Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"978c7a5fa9052b18a7bb72f9","created_at":"2026-09-05T11:57:48.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-6bd19fc562f1436c","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-978c7a5fa9052b18a7bb72f9: done: 0.020566 sec","job_status":"done","queue_duration_s":0.600927,"scheduling_latency_s":0.600943,"redis_calls":10,"redis_duration_s":0.0024619999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001417,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020566,"completed_at":"2026-09-05T11:57:48.906Z","load_balancing_strategy":"primary","db_duration_s":0.002455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.253158] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.908Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"99d8fee1820066140a4c2b3e","created_at":"2026-09-05T11:57:48.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-6bd19fc562f1436c","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-99d8fee1820066140a4c2b3e: start","job_status":"start","queue_duration_s":0.623294,"scheduling_latency_s":0.62331} gitlab # [ 242.265840] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.919Z","message":"No existing merge request to be cleaned up."} gitlab # [ 242.267469] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.928Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"99d8fee1820066140a4c2b3e","created_at":"2026-09-05T11:57:48.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-6bd19fc562f1436c","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-99d8fee1820066140a4c2b3e: done: 0.02028 sec","job_status":"done","queue_duration_s":0.623294,"scheduling_latency_s":0.62331,"redis_calls":10,"redis_duration_s":0.0025180000000000003,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001476,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02028,"completed_at":"2026-09-05T11:57:48.928Z","load_balancing_strategy":"primary","db_duration_s":0.002389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.305939] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.930Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6304ff5485414a0152cd4767","created_at":"2026-09-05T11:57:48.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-6bd19fc562f1436c","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6304ff5485414a0152cd4767: start","job_status":"start","queue_duration_s":0.644948,"scheduling_latency_s":0.644964} gitlab # [ 242.317987] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.941Z","message":"No existing merge request to be cleaned up."} gitlab # [ 242.319626] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.950Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6304ff5485414a0152cd4767","created_at":"2026-09-05T11:57:48.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-6bd19fc562f1436c","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1371bef91c55a8fc8d136d315c578d95","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6304ff5485414a0152cd4767: done: 0.020076 sec","job_status":"done","queue_duration_s":0.644948,"scheduling_latency_s":0.644964,"redis_calls":10,"redis_duration_s":0.002467,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001433,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020076,"completed_at":"2026-09-05T11:57:48.950Z","load_balancing_strategy":"primary","db_duration_s":0.002496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.358313] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62e75c667d9e8ad99a4d9190","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62e75c667d9e8ad99a4d9190: start","job_status":"start","queue_duration_s":0.405682,"scheduling_latency_s":0.405698} gitlab # [ 242.370364] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62e75c667d9e8ad99a4d9190","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62e75c667d9e8ad99a4d9190: done: 0.022371 sec","job_status":"done","queue_duration_s":0.405682,"scheduling_latency_s":0.405698,"redis_calls":16,"redis_duration_s":0.003729,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002108,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003082,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022371,"completed_at":"2026-09-05T11:57:48.973Z","load_balancing_strategy":"primary","db_duration_s":0.0007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.410611] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26177944530b5f23feffd042","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26177944530b5f23feffd042: start","job_status":"start","queue_duration_s":0.429359,"scheduling_latency_s":0.429375} gitlab # [ 242.422982] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26177944530b5f23feffd042","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26177944530b5f23feffd042: done: 0.015289 sec","job_status":"done","queue_duration_s":0.429359,"scheduling_latency_s":0.429375,"redis_calls":13,"redis_duration_s":0.002715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015289,"completed_at":"2026-09-05T11:57:48.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.463033] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:48.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d7895b8cf8fed2e6bfc3685","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d7895b8cf8fed2e6bfc3685: start","job_status":"start","queue_duration_s":0.445877,"scheduling_latency_s":0.445892} gitlab # [ 242.475317] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d7895b8cf8fed2e6bfc3685","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d7895b8cf8fed2e6bfc3685: done: 0.017962 sec","job_status":"done","queue_duration_s":0.445877,"scheduling_latency_s":0.445892,"redis_calls":13,"redis_duration_s":0.002802,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017962,"completed_at":"2026-09-05T11:57:49.009Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.514122] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1f90bdd8ad6c44fa35b4277","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1f90bdd8ad6c44fa35b4277: start","job_status":"start","queue_duration_s":0.465422,"scheduling_latency_s":0.465438} gitlab # [ 242.526323] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1f90bdd8ad6c44fa35b4277","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1f90bdd8ad6c44fa35b4277: done: 0.015358 sec","job_status":"done","queue_duration_s":0.465422,"scheduling_latency_s":0.465438,"redis_calls":13,"redis_duration_s":0.002847,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015358,"completed_at":"2026-09-05T11:57:49.026Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.564947] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9b373980ffc798b962de55f","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9b373980ffc798b962de55f: start","job_status":"start","queue_duration_s":0.48219,"scheduling_latency_s":0.482205} gitlab # [ 242.577252] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9b373980ffc798b962de55f","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9b373980ffc798b962de55f: done: 0.015929 sec","job_status":"done","queue_duration_s":0.48219,"scheduling_latency_s":0.482205,"redis_calls":13,"redis_duration_s":0.00328,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001335,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015929,"completed_at":"2026-09-05T11:57:49.043Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.616028] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd514e66e4b0e8518c2e77ff","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd514e66e4b0e8518c2e77ff: start","job_status":"start","queue_duration_s":0.499418,"scheduling_latency_s":0.499434} gitlab # [ 242.628103] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd514e66e4b0e8518c2e77ff","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd514e66e4b0e8518c2e77ff: done: 0.015389 sec","job_status":"done","queue_duration_s":0.499418,"scheduling_latency_s":0.499434,"redis_calls":13,"redis_duration_s":0.002713,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001772,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015389,"completed_at":"2026-09-05T11:57:49.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.667719] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"935412fa9f33106316f13d97","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-935412fa9f33106316f13d97: start","job_status":"start","queue_duration_s":0.516152,"scheduling_latency_s":0.516167} gitlab # [ 242.680175] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"935412fa9f33106316f13d97","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-935412fa9f33106316f13d97: done: 0.015296 sec","job_status":"done","queue_duration_s":0.516152,"scheduling_latency_s":0.516167,"redis_calls":13,"redis_duration_s":0.00258,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001686,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015296,"completed_at":"2026-09-05T11:57:49.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.718827] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3512269cd1688ce0d49a03f0","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3512269cd1688ce0d49a03f0: start","job_status":"start","queue_duration_s":0.53274,"scheduling_latency_s":0.532755} gitlab # [ 242.731207] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3512269cd1688ce0d49a03f0","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3512269cd1688ce0d49a03f0: done: 0.014855 sec","job_status":"done","queue_duration_s":0.53274,"scheduling_latency_s":0.532755,"redis_calls":13,"redis_duration_s":0.002675,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001617,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014855,"completed_at":"2026-09-05T11:57:49.093Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.770063] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1942dad8c0e7dbf08b6304a","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1942dad8c0e7dbf08b6304a: start","job_status":"start","queue_duration_s":0.548969,"scheduling_latency_s":0.548985} gitlab # [ 242.782280] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1942dad8c0e7dbf08b6304a","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1942dad8c0e7dbf08b6304a: done: 0.015088 sec","job_status":"done","queue_duration_s":0.548969,"scheduling_latency_s":0.548985,"redis_calls":13,"redis_duration_s":0.002755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015088,"completed_at":"2026-09-05T11:57:49.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.820146] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa420e9e481d7e12aef6e21d","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa420e9e481d7e12aef6e21d: start","job_status":"start","queue_duration_s":0.565581,"scheduling_latency_s":0.565598} gitlab # [ 242.832250] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa420e9e481d7e12aef6e21d","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa420e9e481d7e12aef6e21d: done: 0.015217 sec","job_status":"done","queue_duration_s":0.565581,"scheduling_latency_s":0.565598,"redis_calls":13,"redis_duration_s":0.002561,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001651,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015217,"completed_at":"2026-09-05T11:57:49.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.871498] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2908f9546b9e2817f41811dc","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2908f9546b9e2817f41811dc: start","job_status":"start","queue_duration_s":0.582243,"scheduling_latency_s":0.582259} gitlab # [ 242.883751] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2908f9546b9e2817f41811dc","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2908f9546b9e2817f41811dc: done: 0.015618 sec","job_status":"done","queue_duration_s":0.582243,"scheduling_latency_s":0.582259,"redis_calls":13,"redis_duration_s":0.003364,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001683,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015618,"completed_at":"2026-09-05T11:57:49.143Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.921852] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8ae79c5ef8b63b26deacc72","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8ae79c5ef8b63b26deacc72: start","job_status":"start","queue_duration_s":0.599345,"scheduling_latency_s":0.599362} gitlab # [ 242.934037] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8ae79c5ef8b63b26deacc72","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8ae79c5ef8b63b26deacc72: done: 0.015325 sec","job_status":"done","queue_duration_s":0.599345,"scheduling_latency_s":0.599362,"redis_calls":13,"redis_duration_s":0.00261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001653,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015325,"completed_at":"2026-09-05T11:57:49.160Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.971856] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab41692912d78a46ce536a2a","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab41692912d78a46ce536a2a: start","job_status":"start","queue_duration_s":0.616117,"scheduling_latency_s":0.616135} gitlab # [ 242.984281] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab41692912d78a46ce536a2a","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab41692912d78a46ce536a2a: done: 0.015113 sec","job_status":"done","queue_duration_s":0.616117,"scheduling_latency_s":0.616135,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015113,"completed_at":"2026-09-05T11:57:49.176Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.022836] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ef50d407c65d99afc03219b","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ef50d407c65d99afc03219b: start","job_status":"start","queue_duration_s":0.632499,"scheduling_latency_s":0.632516} gitlab # [ 243.035355] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ef50d407c65d99afc03219b","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ef50d407c65d99afc03219b: done: 0.014892 sec","job_status":"done","queue_duration_s":0.632499,"scheduling_latency_s":0.632516,"redis_calls":13,"redis_duration_s":0.001399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014892,"completed_at":"2026-09-05T11:57:49.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.075914] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fee6134ec640137286b5a181","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fee6134ec640137286b5a181: start","job_status":"start","queue_duration_s":0.648598,"scheduling_latency_s":0.648614} gitlab # [ 243.088488] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fee6134ec640137286b5a181","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fee6134ec640137286b5a181: done: 0.014866 sec","job_status":"done","queue_duration_s":0.648598,"scheduling_latency_s":0.648614,"redis_calls":13,"redis_duration_s":0.0017000000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014866,"completed_at":"2026-09-05T11:57:49.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.127798] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8303817bb2a97ec958989c26","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8303817bb2a97ec958989c26: start","job_status":"start","queue_duration_s":0.664515,"scheduling_latency_s":0.664532} gitlab # [ 243.140478] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8303817bb2a97ec958989c26","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8303817bb2a97ec958989c26: done: 0.014456 sec","job_status":"done","queue_duration_s":0.664515,"scheduling_latency_s":0.664532,"redis_calls":13,"redis_duration_s":0.0013670000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000876,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014456,"completed_at":"2026-09-05T11:57:49.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.179968] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3f2374141010cba930a01d8","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3f2374141010cba930a01d8: start","job_status":"start","queue_duration_s":0.680153,"scheduling_latency_s":0.680169} gitlab # [ 243.192547] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3f2374141010cba930a01d8","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3f2374141010cba930a01d8: done: 0.014778 sec","job_status":"done","queue_duration_s":0.680153,"scheduling_latency_s":0.680169,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014778,"completed_at":"2026-09-05T11:57:49.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.231688] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bcaf534a61e5dd6379b5b35","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bcaf534a61e5dd6379b5b35: start","job_status":"start","queue_duration_s":0.695991,"scheduling_latency_s":0.696008} gitlab # [ 243.244112] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bcaf534a61e5dd6379b5b35","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bcaf534a61e5dd6379b5b35: done: 0.014464 sec","job_status":"done","queue_duration_s":0.695991,"scheduling_latency_s":0.696008,"redis_calls":13,"redis_duration_s":0.001386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014464,"completed_at":"2026-09-05T11:57:49.256Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.284748] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce0b538ada091344b7190599","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce0b538ada091344b7190599: start","job_status":"start","queue_duration_s":0.711506,"scheduling_latency_s":0.711523} gitlab # [ 243.321595] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce0b538ada091344b7190599","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce0b538ada091344b7190599: done: 0.014544 sec","job_status":"done","queue_duration_s":0.711506,"scheduling_latency_s":0.711523,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014544,"completed_at":"2026-09-05T11:57:49.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.402972] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d84a3a77f84141693a3ed28","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d84a3a77f84141693a3ed28: start","job_status":"start","queue_duration_s":0.727077,"scheduling_latency_s":0.727093} gitlab # [ 243.424154] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d84a3a77f84141693a3ed28","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d84a3a77f84141693a3ed28: done: 0.014289 sec","job_status":"done","queue_duration_s":0.727077,"scheduling_latency_s":0.727093,"redis_calls":13,"redis_duration_s":0.001364,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000848,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014289,"completed_at":"2026-09-05T11:57:49.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.475519] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"346e234af6976569d46a187e","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-346e234af6976569d46a187e: start","job_status":"start","queue_duration_s":0.74285,"scheduling_latency_s":0.742866} gitlab # [ 243.488883] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"346e234af6976569d46a187e","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-346e234af6976569d46a187e: done: 0.014038 sec","job_status":"done","queue_duration_s":0.74285,"scheduling_latency_s":0.742866,"redis_calls":13,"redis_duration_s":0.001366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014038,"completed_at":"2026-09-05T11:57:49.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.529077] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebae4072811fa444bca045f0","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebae4072811fa444bca045f0: start","job_status":"start","queue_duration_s":0.758102,"scheduling_latency_s":0.758119} gitlab # [ 243.540999] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebae4072811fa444bca045f0","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebae4072811fa444bca045f0: done: 0.01574 sec","job_status":"done","queue_duration_s":0.758102,"scheduling_latency_s":0.758119,"redis_calls":13,"redis_duration_s":0.001731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01574,"completed_at":"2026-09-05T11:57:49.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.579839] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08253dd14a8ac25642491c48","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08253dd14a8ac25642491c48: start","job_status":"start","queue_duration_s":0.775114,"scheduling_latency_s":0.775131} gitlab # [ 243.592265] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08253dd14a8ac25642491c48","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08253dd14a8ac25642491c48: done: 0.014494 sec","job_status":"done","queue_duration_s":0.775114,"scheduling_latency_s":0.775131,"redis_calls":13,"redis_duration_s":0.001373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014494,"completed_at":"2026-09-05T11:57:49.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.631300] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44212ed6186b156d732a05cd","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44212ed6186b156d732a05cd: start","job_status":"start","queue_duration_s":0.790841,"scheduling_latency_s":0.790857} gitlab # [ 243.643773] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44212ed6186b156d732a05cd","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44212ed6186b156d732a05cd: done: 0.014479 sec","job_status":"done","queue_duration_s":0.790841,"scheduling_latency_s":0.790857,"redis_calls":13,"redis_duration_s":0.0013830000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014479,"completed_at":"2026-09-05T11:57:49.351Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.683706] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af658306dc28f15b69175d07","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af658306dc28f15b69175d07: start","job_status":"start","queue_duration_s":0.806574,"scheduling_latency_s":0.806592} gitlab # [ 243.696291] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af658306dc28f15b69175d07","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af658306dc28f15b69175d07: done: 0.014234 sec","job_status":"done","queue_duration_s":0.806574,"scheduling_latency_s":0.806592,"redis_calls":13,"redis_duration_s":0.0014130000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014234,"completed_at":"2026-09-05T11:57:49.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.735184] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7c7d897566dad49a2d0b847","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7c7d897566dad49a2d0b847: start","job_status":"start","queue_duration_s":0.821934,"scheduling_latency_s":0.821951} gitlab # [ 243.747745] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7c7d897566dad49a2d0b847","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7c7d897566dad49a2d0b847: done: 0.014709 sec","job_status":"done","queue_duration_s":0.821934,"scheduling_latency_s":0.821951,"redis_calls":13,"redis_duration_s":0.001386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014709,"completed_at":"2026-09-05T11:57:49.382Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.786559] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69346f024a50de45b3c2db10","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69346f024a50de45b3c2db10: start","job_status":"start","queue_duration_s":0.837694,"scheduling_latency_s":0.837711} gitlab # [ 243.799010] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69346f024a50de45b3c2db10","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69346f024a50de45b3c2db10: done: 0.014155 sec","job_status":"done","queue_duration_s":0.837694,"scheduling_latency_s":0.837711,"redis_calls":13,"redis_duration_s":0.001356,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014155,"completed_at":"2026-09-05T11:57:49.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.838363] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f231049b17dcfe2f1fb76f76","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f231049b17dcfe2f1fb76f76: start","job_status":"start","queue_duration_s":0.852875,"scheduling_latency_s":0.852891} gitlab # [ 243.875078] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f231049b17dcfe2f1fb76f76","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f231049b17dcfe2f1fb76f76: done: 0.014322 sec","job_status":"done","queue_duration_s":0.852875,"scheduling_latency_s":0.852891,"redis_calls":13,"redis_duration_s":0.0014030000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014322,"completed_at":"2026-09-05T11:57:49.412Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.953269] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1736974d1d778f1dea09ab0a","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1736974d1d778f1dea09ab0a: start","job_status":"start","queue_duration_s":0.868212,"scheduling_latency_s":0.868227} gitlab # [ 243.973487] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1736974d1d778f1dea09ab0a","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1736974d1d778f1dea09ab0a: done: 0.014686 sec","job_status":"done","queue_duration_s":0.868212,"scheduling_latency_s":0.868227,"redis_calls":13,"redis_duration_s":0.001376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014686,"completed_at":"2026-09-05T11:57:49.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.019273] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7ae64e959c1b2f50b39f3d8","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7ae64e959c1b2f50b39f3d8: start","job_status":"start","queue_duration_s":0.88419,"scheduling_latency_s":0.884206} gitlab # [ 244.031508] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7ae64e959c1b2f50b39f3d8","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7ae64e959c1b2f50b39f3d8: done: 0.0149 sec","job_status":"done","queue_duration_s":0.88419,"scheduling_latency_s":0.884206,"redis_calls":13,"redis_duration_s":0.0015930000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0149,"completed_at":"2026-09-05T11:57:49.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.069952] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21b6b958ec9ac4c24ab7f778","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21b6b958ec9ac4c24ab7f778: start","job_status":"start","queue_duration_s":0.900117,"scheduling_latency_s":0.900133} gitlab # [ 244.081925] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21b6b958ec9ac4c24ab7f778","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21b6b958ec9ac4c24ab7f778: done: 0.014431 sec","job_status":"done","queue_duration_s":0.900117,"scheduling_latency_s":0.900133,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014431,"completed_at":"2026-09-05T11:57:49.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.120074] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d5aa95f6c56252c3c2cfc06","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d5aa95f6c56252c3c2cfc06: start","job_status":"start","queue_duration_s":0.91581,"scheduling_latency_s":0.915826} gitlab # [ 244.133332] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d5aa95f6c56252c3c2cfc06","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d5aa95f6c56252c3c2cfc06: done: 0.014123 sec","job_status":"done","queue_duration_s":0.91581,"scheduling_latency_s":0.915826,"redis_calls":13,"redis_duration_s":0.0013540000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000848,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014123,"completed_at":"2026-09-05T11:57:49.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.171663] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1456b4da611a1fa0f0edc00d","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1456b4da611a1fa0f0edc00d: start","job_status":"start","queue_duration_s":0.931269,"scheduling_latency_s":0.931285} gitlab # [ 244.184135] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1456b4da611a1fa0f0edc00d","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1456b4da611a1fa0f0edc00d: done: 0.014541 sec","job_status":"done","queue_duration_s":0.931269,"scheduling_latency_s":0.931285,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000688,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014541,"completed_at":"2026-09-05T11:57:49.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.222091] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bd9adb658989ce7ef29d323","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bd9adb658989ce7ef29d323: start","job_status":"start","queue_duration_s":0.947071,"scheduling_latency_s":0.94709} gitlab # [ 244.234132] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bd9adb658989ce7ef29d323","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bd9adb658989ce7ef29d323: done: 0.014307 sec","job_status":"done","queue_duration_s":0.947071,"scheduling_latency_s":0.94709,"redis_calls":13,"redis_duration_s":0.0015700000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014307,"completed_at":"2026-09-05T11:57:49.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.272387] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a7e87a34bb2f195dc555ef","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a7e87a34bb2f195dc555ef: start","job_status":"start","queue_duration_s":0.962688,"scheduling_latency_s":0.962705} gitlab # [ 244.284521] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a7e87a34bb2f195dc555ef","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a7e87a34bb2f195dc555ef: done: 0.014423 sec","job_status":"done","queue_duration_s":0.962688,"scheduling_latency_s":0.962705,"redis_calls":13,"redis_duration_s":0.001413,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014423,"completed_at":"2026-09-05T11:57:49.522Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.322804] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"079d98bd96e9cc36087bfe14","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-079d98bd96e9cc36087bfe14: start","job_status":"start","queue_duration_s":0.978265,"scheduling_latency_s":0.978287} gitlab # [ 244.335915] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"079d98bd96e9cc36087bfe14","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-079d98bd96e9cc36087bfe14: done: 0.014945 sec","job_status":"done","queue_duration_s":0.978265,"scheduling_latency_s":0.978287,"redis_calls":13,"redis_duration_s":0.001367,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014945,"completed_at":"2026-09-05T11:57:49.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.373815] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b970edd48ecc5e60f5b8e0b9","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b970edd48ecc5e60f5b8e0b9: start","job_status":"start","queue_duration_s":0.994269,"scheduling_latency_s":0.994284} gitlab # [ 244.385874] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b970edd48ecc5e60f5b8e0b9","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b970edd48ecc5e60f5b8e0b9: done: 0.014353 sec","job_status":"done","queue_duration_s":0.994269,"scheduling_latency_s":0.994284,"redis_calls":13,"redis_duration_s":0.001373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014353,"completed_at":"2026-09-05T11:57:49.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.423725] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"976e107d1d60fc91f6ee12cf","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-976e107d1d60fc91f6ee12cf: start","job_status":"start","queue_duration_s":1.009665,"scheduling_latency_s":1.009687} gitlab # [ 244.436127] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"976e107d1d60fc91f6ee12cf","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-976e107d1d60fc91f6ee12cf: done: 0.014499 sec","job_status":"done","queue_duration_s":1.009665,"scheduling_latency_s":1.009687,"redis_calls":13,"redis_duration_s":0.001405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014499,"completed_at":"2026-09-05T11:57:49.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.473996] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61f583578604110a555544ae","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61f583578604110a555544ae: start","job_status":"start","queue_duration_s":1.025199,"scheduling_latency_s":1.025214} gitlab # [ 244.486258] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61f583578604110a555544ae","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61f583578604110a555544ae: done: 0.014508 sec","job_status":"done","queue_duration_s":1.025199,"scheduling_latency_s":1.025214,"redis_calls":13,"redis_duration_s":0.001403,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014508,"completed_at":"2026-09-05T11:57:49.585Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.523860] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5bcf674e9a1355eb4dc2aba","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5bcf674e9a1355eb4dc2aba: start","job_status":"start","queue_duration_s":1.040734,"scheduling_latency_s":1.040749} gitlab # [ 244.536961] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5bcf674e9a1355eb4dc2aba","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5bcf674e9a1355eb4dc2aba: done: 0.01455 sec","job_status":"done","queue_duration_s":1.040734,"scheduling_latency_s":1.040749,"redis_calls":13,"redis_duration_s":0.001558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01455,"completed_at":"2026-09-05T11:57:49.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.574817] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e2cd9703a17bcd0626d2362","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e2cd9703a17bcd0626d2362: start","job_status":"start","queue_duration_s":1.056353,"scheduling_latency_s":1.05637} gitlab # [ 244.586918] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e2cd9703a17bcd0626d2362","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e2cd9703a17bcd0626d2362: done: 0.028363 sec","job_status":"done","queue_duration_s":1.056353,"scheduling_latency_s":1.05637,"redis_calls":13,"redis_duration_s":0.002729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028363,"completed_at":"2026-09-05T11:57:49.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.624875] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd31924da730dee8e8965fb3","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd31924da730dee8e8965fb3: start","job_status":"start","queue_duration_s":1.088071,"scheduling_latency_s":1.088114} gitlab # [ 244.636836] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd31924da730dee8e8965fb3","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd31924da730dee8e8965fb3: done: 0.036671 sec","job_status":"done","queue_duration_s":1.088071,"scheduling_latency_s":1.088114,"redis_calls":13,"redis_duration_s":0.0032700000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036671,"completed_at":"2026-09-05T11:57:49.670Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.675183] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07c3376630ea570f79b965c7","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07c3376630ea570f79b965c7: start","job_status":"start","queue_duration_s":1.127381,"scheduling_latency_s":1.127415} gitlab # [ 244.687842] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.703Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07c3376630ea570f79b965c7","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07c3376630ea570f79b965c7: done: 0.030627 sec","job_status":"done","queue_duration_s":1.127381,"scheduling_latency_s":1.127415,"redis_calls":13,"redis_duration_s":0.0035919999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002335,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030627,"completed_at":"2026-09-05T11:57:49.703Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.727081] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ec6de61eb527ba5be916ade","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ec6de61eb527ba5be916ade: start","job_status":"start","queue_duration_s":1.160045,"scheduling_latency_s":1.160071} gitlab # [ 244.740633] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ec6de61eb527ba5be916ade","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ec6de61eb527ba5be916ade: done: 0.022377 sec","job_status":"done","queue_duration_s":1.160045,"scheduling_latency_s":1.160071,"redis_calls":13,"redis_duration_s":0.003175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022377,"completed_at":"2026-09-05T11:57:49.728Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.779576] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ceb49da3c908940b51abaed","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ceb49da3c908940b51abaed: start","job_status":"start","queue_duration_s":1.184114,"scheduling_latency_s":1.184134} gitlab # [ 244.792131] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ceb49da3c908940b51abaed","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ceb49da3c908940b51abaed: done: 0.019088 sec","job_status":"done","queue_duration_s":1.184114,"scheduling_latency_s":1.184134,"redis_calls":13,"redis_duration_s":0.003074,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019088,"completed_at":"2026-09-05T11:57:49.748Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.830792] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06730a714a16d0edffa260bc","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06730a714a16d0edffa260bc: start","job_status":"start","queue_duration_s":1.204891,"scheduling_latency_s":1.20491} gitlab # [ 244.843007] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06730a714a16d0edffa260bc","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06730a714a16d0edffa260bc: done: 0.017804 sec","job_status":"done","queue_duration_s":1.204891,"scheduling_latency_s":1.20491,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001842,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017804,"completed_at":"2026-09-05T11:57:49.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.881809] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6c44b6c1e3224d85f78ca87","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6c44b6c1e3224d85f78ca87: start","job_status":"start","queue_duration_s":1.224312,"scheduling_latency_s":1.22433} gitlab # [ 244.894134] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6c44b6c1e3224d85f78ca87","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6c44b6c1e3224d85f78ca87: done: 0.017338 sec","job_status":"done","queue_duration_s":1.224312,"scheduling_latency_s":1.22433,"redis_calls":13,"redis_duration_s":0.002713,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017338,"completed_at":"2026-09-05T11:57:49.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.934246] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d162e1c6037be7246dc5be5a","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d162e1c6037be7246dc5be5a: start","job_status":"start","queue_duration_s":1.243225,"scheduling_latency_s":1.243249} gitlab # [ 244.947451] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d162e1c6037be7246dc5be5a","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d162e1c6037be7246dc5be5a: done: 0.016537 sec","job_status":"done","queue_duration_s":1.243225,"scheduling_latency_s":1.243249,"redis_calls":13,"redis_duration_s":0.002749,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001742,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016537,"completed_at":"2026-09-05T11:57:49.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.987223] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"798c41c7c483288f28896490","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-798c41c7c483288f28896490: start","job_status":"start","queue_duration_s":1.278162,"scheduling_latency_s":1.278181} gitlab # [ 244.999813] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"798c41c7c483288f28896490","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-798c41c7c483288f28896490: done: 0.020653 sec","job_status":"done","queue_duration_s":1.278162,"scheduling_latency_s":1.278181,"redis_calls":13,"redis_duration_s":0.005558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001424,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020653,"completed_at":"2026-09-05T11:57:49.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.038643] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50b313a6f2a05dae27712519","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50b313a6f2a05dae27712519: start","job_status":"start","queue_duration_s":1.300193,"scheduling_latency_s":1.300212} gitlab # [ 245.050920] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:57:49.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50b313a6f2a05dae27712519","created_at":"2026-09-05T11:57:48.465Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-66442390d0b04bc2","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f8f96eebd5dab98cb3275e13801abd4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:57:48.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50b313a6f2a05dae27712519: done: 0.015582 sec","job_status":"done","queue_duration_s":1.300193,"scheduling_latency_s":1.300212,"redis_calls":13,"redis_duration_s":0.002847,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015582,"completed_at":"2026-09-05T11:57:49.861Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.193439] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9e8f9c61d16eab4ac5e86c0a","created_at":"2026-09-05T11:58:01.052Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbbfd0041367f1593b097ebff290e58a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.068Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9e8f9c61d16eab4ac5e86c0a: start","job_status":"start","queue_duration_s":0.005344,"scheduling_latency_s":0.005403} gitlab # [ 254.347280] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9e8f9c61d16eab4ac5e86c0a","created_at":"2026-09-05T11:58:01.052Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbbfd0041367f1593b097ebff290e58a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.068Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9e8f9c61d16eab4ac5e86c0a: done: 0.155692 sec","job_status":"done","queue_duration_s":0.005344,"scheduling_latency_s":0.005403,"redis_calls":7,"redis_duration_s":0.013158,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010524,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002634,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.155692,"completed_at":"2026-09-05T11:58:01.229Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.423437] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.274Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0c3c6f8a7efb794bd42611c6","created_at":"2026-09-05T11:58:01.104Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"adfbebb2c64c73400fef656eef16f10a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.109Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0c3c6f8a7efb794bd42611c6: start","job_status":"start","queue_duration_s":0.164338,"scheduling_latency_s":0.164399} gitlab # [ 254.443033] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.326Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 254.482673] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.363Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0c3c6f8a7efb794bd42611c6","created_at":"2026-09-05T11:58:01.104Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"adfbebb2c64c73400fef656eef16f10a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.109Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0c3c6f8a7efb794bd42611c6: done: 0.089985 sec","job_status":"done","queue_duration_s":0.164338,"scheduling_latency_s":0.164399,"redis_calls":7,"redis_duration_s":0.015208,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01315,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002058,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.089985,"completed_at":"2026-09-05T11:58:01.363Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.569510] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1fa39122f92a6e9c01547964","created_at":"2026-09-05T11:58:01.115Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99b9e0f81089fd0c0a58150934e7635d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.120Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1fa39122f92a6e9c01547964: start","job_status":"start","queue_duration_s":0.252367,"scheduling_latency_s":0.252427} gitlab # [ 254.607248] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.410Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 254.614587] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1fa39122f92a6e9c01547964","created_at":"2026-09-05T11:58:01.115Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99b9e0f81089fd0c0a58150934e7635d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.120Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1fa39122f92a6e9c01547964: done: 0.068405 sec","job_status":"done","queue_duration_s":0.252367,"scheduling_latency_s":0.252427,"redis_calls":7,"redis_duration_s":0.016436,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012388,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004048,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.068405,"completed_at":"2026-09-05T11:58:01.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.669034] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b09b4268806b14bccd05f7ef","created_at":"2026-09-05T11:58:01.149Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edb3b9dc394071efcbaf398e79d5e5ab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.159Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b09b4268806b14bccd05f7ef: start","job_status":"start","queue_duration_s":0.34627,"scheduling_latency_s":0.346308} gitlab # [ 254.821347] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b09b4268806b14bccd05f7ef","created_at":"2026-09-05T11:58:01.149Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edb3b9dc394071efcbaf398e79d5e5ab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.159Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b09b4268806b14bccd05f7ef: done: 0.198028 sec","job_status":"done","queue_duration_s":0.34627,"scheduling_latency_s":0.346308,"redis_calls":9,"redis_duration_s":0.024825999999999997,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.018142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006684,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0020648,"exclusive_lock_hold_duration_s":0.04524574,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.198028,"completed_at":"2026-09-05T11:58:01.703Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.101258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.917066] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.725Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"11aa4bd6006d2679acff3d36","created_at":"2026-09-05T11:58:01.196Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da095ec59d885e832643a0bb889e0804","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.219Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-11aa4bd6006d2679acff3d36: start","job_status":"start","queue_duration_s":0.505356,"scheduling_latency_s":0.505404} gitlab # [ 254.939599] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"11aa4bd6006d2679acff3d36","created_at":"2026-09-05T11:58:01.196Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da095ec59d885e832643a0bb889e0804","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.219Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-11aa4bd6006d2679acff3d36: done: 0.091986 sec","job_status":"done","queue_duration_s":0.505356,"scheduling_latency_s":0.505404,"redis_calls":7,"redis_duration_s":0.025916,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.02291,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003006,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091986,"completed_at":"2026-09-05T11:58:01.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.996070] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:01.822Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"5034d0d6899c83605e2932d7","created_at":"2026-09-05T11:58:01.264Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21da92abad4783c1a072d3630bedc6fb","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.272Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-5034d0d6899c83605e2932d7: start","job_status":"start","queue_duration_s":0.549083,"scheduling_latency_s":0.550409} gitlab # [ 255.487455] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.369Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"5034d0d6899c83605e2932d7","created_at":"2026-09-05T11:58:01.264Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21da92abad4783c1a072d3630bedc6fb","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.272Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-5034d0d6899c83605e2932d7: done: 0.547841 sec","job_status":"done","queue_duration_s":0.549083,"scheduling_latency_s":0.550409,"redis_calls":562,"redis_duration_s":0.180908,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.179759,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.400485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.547841,"completed_at":"2026-09-05T11:58:02.369Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.592088] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"afee56e5c63adff0a3074536","created_at":"2026-09-05T11:58:01.297Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8894a2b22f1d690138576584adb604e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.319Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-afee56e5c63adff0a3074536: start","job_status":"start","queue_duration_s":1.052205,"scheduling_latency_s":1.052236} gitlab # [ 255.615751] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"afee56e5c63adff0a3074536","created_at":"2026-09-05T11:58:01.297Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8894a2b22f1d690138576584adb604e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.319Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-afee56e5c63adff0a3074536: done: 0.018829 sec","job_status":"done","queue_duration_s":1.052205,"scheduling_latency_s":1.052236,"redis_calls":9,"redis_duration_s":0.002379,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005979,"exclusive_lock_hold_duration_s":0.00364272,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.018829,"completed_at":"2026-09-05T11:58:02.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.663904] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.391Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"699f0bf219bbbe2a02fb51d6","created_at":"2026-09-05T11:58:01.345Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29a71de1bab852691a3652fb52186f36","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.357Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-699f0bf219bbbe2a02fb51d6: start","job_status":"start","queue_duration_s":1.033603,"scheduling_latency_s":1.033618} gitlab # [ 255.677910] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.414Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"699f0bf219bbbe2a02fb51d6","created_at":"2026-09-05T11:58:01.345Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29a71de1bab852691a3652fb52186f36","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.357Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-699f0bf219bbbe2a02fb51d6: done: 0.022919 sec","job_status":"done","queue_duration_s":1.033603,"scheduling_latency_s":1.033618,"redis_calls":7,"redis_duration_s":0.0017989999999999998,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001318,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022919,"completed_at":"2026-09-05T11:58:02.414Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.718248] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"97caceed9eba36796521780f","created_at":"2026-09-05T11:58:01.371Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb4121010a5e50eb7e44d926b0613e9e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.379Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-97caceed9eba36796521780f: start","job_status":"start","queue_duration_s":1.036999,"scheduling_latency_s":1.037017} gitlab # [ 255.731612] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.433Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"97caceed9eba36796521780f","created_at":"2026-09-05T11:58:01.371Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb4121010a5e50eb7e44d926b0613e9e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.379Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-97caceed9eba36796521780f: done: 0.017368 sec","job_status":"done","queue_duration_s":1.036999,"scheduling_latency_s":1.037017,"redis_calls":9,"redis_duration_s":0.002284,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001185,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005313,"exclusive_lock_hold_duration_s":0.0030811,"duration_s":0.017368,"completed_at":"2026-09-05T11:58:02.433Z","load_balancing_strategy":"primary","db_duration_s":0.00082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.772988] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1cfbad9165a88af6b5604314","created_at":"2026-09-05T11:58:01.406Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"254cfc712c1eefca8a6ac0e6e7e28f9e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.425Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1cfbad9165a88af6b5604314: start","job_status":"start","queue_duration_s":1.009554,"scheduling_latency_s":1.00957} gitlab # [ 255.787345] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1cfbad9165a88af6b5604314","created_at":"2026-09-05T11:58:01.406Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"254cfc712c1eefca8a6ac0e6e7e28f9e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.425Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1cfbad9165a88af6b5604314: done: 0.022266 sec","job_status":"done","queue_duration_s":1.009554,"scheduling_latency_s":1.00957,"redis_calls":10,"redis_duration_s":0.002711,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001612,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0049345,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.022266,"completed_at":"2026-09-05T11:58:02.457Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002488,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 255.829798] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.459Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b94e660c511f46fcb5ce2e13","created_at":"2026-09-05T11:58:01.500Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5dd68268a5deb7c54f76767ed585e79c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.510Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b94e660c511f46fcb5ce2e13: start","job_status":"start","queue_duration_s":0.949181,"scheduling_latency_s":0.949197} gitlab # [ 255.844341] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.467Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 255.847459] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.472Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b94e660c511f46fcb5ce2e13","created_at":"2026-09-05T11:58:01.500Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5dd68268a5deb7c54f76767ed585e79c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.510Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b94e660c511f46fcb5ce2e13: done: 0.013031 sec","job_status":"done","queue_duration_s":0.949181,"scheduling_latency_s":0.949197,"redis_calls":7,"redis_duration_s":0.001615,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000475,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013031,"completed_at":"2026-09-05T11:58:02.472Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.887950] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.473Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0a63d04f5eb149c5303ec9f3","created_at":"2026-09-05T11:58:01.579Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01f8a7d4c99276223ea4a316e4566440","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.582Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0a63d04f5eb149c5303ec9f3: start","job_status":"start","queue_duration_s":0.89117,"scheduling_latency_s":0.891185} gitlab # [ 255.902052] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0a63d04f5eb149c5303ec9f3","created_at":"2026-09-05T11:58:01.579Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01f8a7d4c99276223ea4a316e4566440","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.582Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0a63d04f5eb149c5303ec9f3: done: 0.01732 sec","job_status":"done","queue_duration_s":0.89117,"scheduling_latency_s":0.891185,"redis_calls":7,"redis_duration_s":0.00179,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.01732,"completed_at":"2026-09-05T11:58:02.490Z","load_balancing_strategy":"primary","db_duration_s":0.002961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.942733] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"41e192decd946375e5e84724","created_at":"2026-09-05T11:58:01.598Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.607Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-41e192decd946375e5e84724: start","job_status":"start","queue_duration_s":0.885534,"scheduling_latency_s":0.88555} gitlab # [ 255.956380] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"41e192decd946375e5e84724","created_at":"2026-09-05T11:58:01.598Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.607Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-41e192decd946375e5e84724: done: 0.025679 sec","job_status":"done","queue_duration_s":0.885534,"scheduling_latency_s":0.88555,"redis_calls":14,"redis_duration_s":0.003311,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000379,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002163,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000769,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025679,"completed_at":"2026-09-05T11:58:02.518Z","load_balancing_strategy":"primary","db_duration_s":0.000852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.998736] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.520Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6c1c012c2f495b081027c76a","created_at":"2026-09-05T11:58:01.621Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6538bab08f395fd43dc8e1f75e178763","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.639Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6c1c012c2f495b081027c76a: start","job_status":"start","queue_duration_s":0.880512,"scheduling_latency_s":0.880529} gitlab # [ 256.013462] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6c1c012c2f495b081027c76a","created_at":"2026-09-05T11:58:01.621Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6538bab08f395fd43dc8e1f75e178763","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.639Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6c1c012c2f495b081027c76a: done: 0.026748 sec","job_status":"done","queue_duration_s":0.880512,"scheduling_latency_s":0.880529,"redis_calls":12,"redis_duration_s":0.003118,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00053,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001208,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00056408,"exclusive_lock_hold_duration_s":0.0072978999999999995,"duration_s":0.026748,"completed_at":"2026-09-05T11:58:02.546Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.055647] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"1b6e3e8715a018ba542a989c","created_at":"2026-09-05T11:58:01.653Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6e703ada8a52e2b1cb45005bb22bd1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.660Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-1b6e3e8715a018ba542a989c: start","job_status":"start","queue_duration_s":0.887544,"scheduling_latency_s":0.887561} gitlab # [ 256.070457] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"1b6e3e8715a018ba542a989c","created_at":"2026-09-05T11:58:01.653Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6e703ada8a52e2b1cb45005bb22bd1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.660Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-1b6e3e8715a018ba542a989c: done: 0.041282 sec","job_status":"done","queue_duration_s":0.887544,"scheduling_latency_s":0.887561,"redis_calls":9,"redis_duration_s":0.002411,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052498,"exclusive_lock_hold_duration_s":0.02670364,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.041282,"completed_at":"2026-09-05T11:58:02.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.113394] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"91ecaf002c325253ef410535","created_at":"2026-09-05T11:58:01.690Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b55471411364baf4ffd226a8c2423ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-05T11:58:01.698Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-91ecaf002c325253ef410535: start","job_status":"start","queue_duration_s":0.892584,"scheduling_latency_s":0.892599} gitlab # [ 256.126902] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.611Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"91ecaf002c325253ef410535","created_at":"2026-09-05T11:58:01.690Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b55471411364baf4ffd226a8c2423ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-05T11:58:01.698Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-91ecaf002c325253ef410535: done: 0.020419 sec","job_status":"done","queue_duration_s":0.892584,"scheduling_latency_s":0.892599,"redis_calls":7,"redis_duration_s":0.001814,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001337,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020419,"completed_at":"2026-09-05T11:58:02.611Z","load_balancing_strategy":"primary","db_duration_s":0.001498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.166037] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.612Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a75e48669fc4ddf7ed6ff2b5","created_at":"2026-09-05T11:58:01.724Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32c9c93faf7a4df3d841ff93fa2cfc87","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-05T11:58:01.732Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a75e48669fc4ddf7ed6ff2b5: start","job_status":"start","queue_duration_s":0.880456,"scheduling_latency_s":0.880473} gitlab # [ 256.179124] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.620Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 256.182865] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.625Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a75e48669fc4ddf7ed6ff2b5","created_at":"2026-09-05T11:58:01.724Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32c9c93faf7a4df3d841ff93fa2cfc87","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-05T11:58:01.732Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a75e48669fc4ddf7ed6ff2b5: done: 0.012715 sec","job_status":"done","queue_duration_s":0.880456,"scheduling_latency_s":0.880473,"redis_calls":7,"redis_duration_s":0.00164,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012715,"completed_at":"2026-09-05T11:58:02.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.221021] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7f162a9a21f686828de80cb0","created_at":"2026-09-05T11:58:01.770Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1b38e991022ad8b38fdee0fa294abf7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.776Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7f162a9a21f686828de80cb0: start","job_status":"start","queue_duration_s":0.851001,"scheduling_latency_s":0.851017} gitlab # [ 256.235233] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7f162a9a21f686828de80cb0","created_at":"2026-09-05T11:58:01.770Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1b38e991022ad8b38fdee0fa294abf7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.776Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7f162a9a21f686828de80cb0: done: 0.013006 sec","job_status":"done","queue_duration_s":0.851001,"scheduling_latency_s":0.851017,"redis_calls":7,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013006,"completed_at":"2026-09-05T11:58:02.640Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.274923] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"1ba1fa5d05dc5518d5a5c971","created_at":"2026-09-05T11:58:01.805Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.814Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-1ba1fa5d05dc5518d5a5c971: start","job_status":"start","queue_duration_s":0.827516,"scheduling_latency_s":0.82753} gitlab # [ 256.288801] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"1ba1fa5d05dc5518d5a5c971","created_at":"2026-09-05T11:58:01.805Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.814Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-1ba1fa5d05dc5518d5a5c971: done: 0.087842 sec","job_status":"done","queue_duration_s":0.827516,"scheduling_latency_s":0.82753,"redis_calls":60,"redis_duration_s":0.014437,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000622,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013141,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000674,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087842,"completed_at":"2026-09-05T11:58:02.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.328376] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:02.730Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ca1e2972d453e0e00c90ca17","created_at":"2026-09-05T11:58:01.829Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8aa644b551dc09fadf052d920150758","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.836Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ca1e2972d453e0e00c90ca17: start","job_status":"start","queue_duration_s":0.894518,"scheduling_latency_s":0.894533} gitlab # [ 256.341839] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ca1e2972d453e0e00c90ca17","created_at":"2026-09-05T11:58:01.829Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8aa644b551dc09fadf052d920150758","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.836Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ca1e2972d453e0e00c90ca17: done: 0.269086 sec","job_status":"done","queue_duration_s":0.894518,"scheduling_latency_s":0.894533,"redis_calls":11,"redis_duration_s":0.002999,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000514,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001291,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.200507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00054314,"exclusive_lock_hold_duration_s":0.25107018,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.269086,"completed_at":"2026-09-05T11:58:02.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.080324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.386214] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3fbab5377b68ee203f945560","created_at":"2026-09-05T11:58:01.840Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"708a74a8d2269772366c11bd88a2a5f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.950Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3fbab5377b68ee203f945560: start","job_status":"start","queue_duration_s":1.050772,"scheduling_latency_s":1.050788} gitlab # [ 256.400538] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.009Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 256.403763] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.014Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3fbab5377b68ee203f945560","created_at":"2026-09-05T11:58:01.840Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"708a74a8d2269772366c11bd88a2a5f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.950Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3fbab5377b68ee203f945560: done: 0.012966 sec","job_status":"done","queue_duration_s":1.050772,"scheduling_latency_s":1.050788,"redis_calls":7,"redis_duration_s":0.001728,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001254,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012966,"completed_at":"2026-09-05T11:58:03.014Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.443825] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.016Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"911b401ddba65eef6aca8567","created_at":"2026-09-05T11:58:01.973Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"155b3cebdc01f319d0f7eb2000cea791","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.990Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-911b401ddba65eef6aca8567: start","job_status":"start","queue_duration_s":1.02527,"scheduling_latency_s":1.025286} gitlab # [ 256.457206] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.034Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"911b401ddba65eef6aca8567","created_at":"2026-09-05T11:58:01.973Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"155b3cebdc01f319d0f7eb2000cea791","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:01.990Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-911b401ddba65eef6aca8567: done: 0.018761 sec","job_status":"done","queue_duration_s":1.02527,"scheduling_latency_s":1.025286,"redis_calls":12,"redis_duration_s":0.002895,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000601,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000514,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003183,"duration_s":0.018761,"completed_at":"2026-09-05T11:58:03.034Z","load_balancing_strategy":"primary","db_duration_s":0.000639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.499288] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"eab06e19dd94db469a8bad0f","created_at":"2026-09-05T11:58:02.013Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"001eb69c8d91e0af4c0325b37cec3f7b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-05T11:58:02.022Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-eab06e19dd94db469a8bad0f: start","job_status":"start","queue_duration_s":1.01437,"scheduling_latency_s":1.014385} gitlab # [ 256.512586] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.044Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 256.515790] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"eab06e19dd94db469a8bad0f","created_at":"2026-09-05T11:58:02.013Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"001eb69c8d91e0af4c0325b37cec3f7b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-05T11:58:02.022Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-eab06e19dd94db469a8bad0f: done: 0.01297 sec","job_status":"done","queue_duration_s":1.01437,"scheduling_latency_s":1.014385,"redis_calls":7,"redis_duration_s":0.002097,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001652,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01297,"completed_at":"2026-09-05T11:58:03.049Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.554689] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"56d80929636299ed16d9896d","created_at":"2026-09-05T11:58:02.041Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98f747f9c2cc094a4ee58271eec2a19a","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.063Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-56d80929636299ed16d9896d: start","job_status":"start","queue_duration_s":0.987693,"scheduling_latency_s":0.987709} gitlab # [ 256.569146] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"56d80929636299ed16d9896d","created_at":"2026-09-05T11:58:02.041Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98f747f9c2cc094a4ee58271eec2a19a","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.063Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-56d80929636299ed16d9896d: done: 0.028671 sec","job_status":"done","queue_duration_s":0.987693,"scheduling_latency_s":0.987709,"redis_calls":10,"redis_duration_s":0.002787,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001668,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000571,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046882,"duration_s":0.028671,"completed_at":"2026-09-05T11:58:03.079Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.612108] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.080Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"974a6755f33ee26e950a7583","created_at":"2026-09-05T11:58:02.502Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-db0178dd264142e8","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.512Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-974a6755f33ee26e950a7583: start","job_status":"start","queue_duration_s":0.568146,"scheduling_latency_s":0.568162} gitlab # [ 256.624164] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.092Z","message":"No existing merge request to be cleaned up."} gitlab # [ 256.625807] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.100Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"974a6755f33ee26e950a7583","created_at":"2026-09-05T11:58:02.502Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-db0178dd264142e8","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.512Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-974a6755f33ee26e950a7583: done: 0.019871 sec","job_status":"done","queue_duration_s":0.568146,"scheduling_latency_s":0.568162,"redis_calls":10,"redis_duration_s":0.002506,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001037,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001469,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019871,"completed_at":"2026-09-05T11:58:03.100Z","load_balancing_strategy":"primary","db_duration_s":0.002535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.663586] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.102Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e83f3bd8244b01f3b26d3781","created_at":"2026-09-05T11:58:02.502Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-db0178dd264142e8","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.512Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e83f3bd8244b01f3b26d3781: start","job_status":"start","queue_duration_s":0.589447,"scheduling_latency_s":0.589461} gitlab # [ 256.675864] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.113Z","message":"No existing merge request to be cleaned up."} gitlab # [ 256.677635] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.122Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e83f3bd8244b01f3b26d3781","created_at":"2026-09-05T11:58:02.502Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-db0178dd264142e8","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.512Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e83f3bd8244b01f3b26d3781: done: 0.019831 sec","job_status":"done","queue_duration_s":0.589447,"scheduling_latency_s":0.589461,"redis_calls":10,"redis_duration_s":0.002899,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001213,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001686,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019831,"completed_at":"2026-09-05T11:58:03.121Z","load_balancing_strategy":"primary","db_duration_s":0.002689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.715483] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.123Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fcc4808ffe3631556474376f","created_at":"2026-09-05T11:58:02.502Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-db0178dd264142e8","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.512Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fcc4808ffe3631556474376f: start","job_status":"start","queue_duration_s":0.610705,"scheduling_latency_s":0.61072} gitlab # [ 256.727725] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.134Z","message":"No existing merge request to be cleaned up."} gitlab # [ 256.729453] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.142Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fcc4808ffe3631556474376f","created_at":"2026-09-05T11:58:02.502Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-db0178dd264142e8","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.512Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fcc4808ffe3631556474376f: done: 0.019278 sec","job_status":"done","queue_duration_s":0.610705,"scheduling_latency_s":0.61072,"redis_calls":10,"redis_duration_s":0.002435,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019278,"completed_at":"2026-09-05T11:58:03.142Z","load_balancing_strategy":"primary","db_duration_s":0.002389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.767353] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.144Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"66305cf3117800af5c9e27d5","created_at":"2026-09-05T11:58:02.502Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-db0178dd264142e8","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.512Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-66305cf3117800af5c9e27d5: start","job_status":"start","queue_duration_s":0.631952,"scheduling_latency_s":0.631969} gitlab # [ 256.780700] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.155Z","message":"No existing merge request to be cleaned up."} gitlab # [ 256.782334] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.163Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"66305cf3117800af5c9e27d5","created_at":"2026-09-05T11:58:02.502Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-db0178dd264142e8","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"75f0ef452b2465b5620973b6fab79e71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.512Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-66305cf3117800af5c9e27d5: done: 0.019242 sec","job_status":"done","queue_duration_s":0.631952,"scheduling_latency_s":0.631969,"redis_calls":10,"redis_duration_s":0.002565,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001503,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019242,"completed_at":"2026-09-05T11:58:03.163Z","load_balancing_strategy":"primary","db_duration_s":0.002368,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.820017] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4478ed1dcfa7cc014f621dc","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4478ed1dcfa7cc014f621dc: start","job_status":"start","queue_duration_s":0.442864,"scheduling_latency_s":0.44288} gitlab # [ 256.832135] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4478ed1dcfa7cc014f621dc","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4478ed1dcfa7cc014f621dc: done: 0.015753 sec","job_status":"done","queue_duration_s":0.442864,"scheduling_latency_s":0.44288,"redis_calls":13,"redis_duration_s":0.002445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001725,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015753,"completed_at":"2026-09-05T11:58:03.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.870424] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f691a3f5c06ffd1c19c55d8","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f691a3f5c06ffd1c19c55d8: start","job_status":"start","queue_duration_s":0.459672,"scheduling_latency_s":0.459687} gitlab # [ 256.882661] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f691a3f5c06ffd1c19c55d8","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f691a3f5c06ffd1c19c55d8: done: 0.013582 sec","job_status":"done","queue_duration_s":0.459672,"scheduling_latency_s":0.459687,"redis_calls":13,"redis_duration_s":0.001392,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013582,"completed_at":"2026-09-05T11:58:03.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.921142] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad0bd7008aa79046504318de","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad0bd7008aa79046504318de: start","job_status":"start","queue_duration_s":0.474325,"scheduling_latency_s":0.47434} gitlab # [ 256.933136] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad0bd7008aa79046504318de","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad0bd7008aa79046504318de: done: 0.013772 sec","job_status":"done","queue_duration_s":0.474325,"scheduling_latency_s":0.47434,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013772,"completed_at":"2026-09-05T11:58:03.210Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.971563] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f638d13c56ab313c67225ba","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f638d13c56ab313c67225ba: start","job_status":"start","queue_duration_s":0.489171,"scheduling_latency_s":0.489188} gitlab # [ 256.983935] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f638d13c56ab313c67225ba","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f638d13c56ab313c67225ba: done: 0.013477 sec","job_status":"done","queue_duration_s":0.489171,"scheduling_latency_s":0.489188,"redis_calls":13,"redis_duration_s":0.0013540000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013477,"completed_at":"2026-09-05T11:58:03.225Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.024393] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ef5eb46d64533483193c3ea","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ef5eb46d64533483193c3ea: start","job_status":"start","queue_duration_s":0.503659,"scheduling_latency_s":0.503674} gitlab # [ 257.036856] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ef5eb46d64533483193c3ea","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ef5eb46d64533483193c3ea: done: 0.013519 sec","job_status":"done","queue_duration_s":0.503659,"scheduling_latency_s":0.503674,"redis_calls":13,"redis_duration_s":0.0013440000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000848,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013519,"completed_at":"2026-09-05T11:58:03.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.075424] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5fbfe04fc769b2a1b728e42","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5fbfe04fc769b2a1b728e42: start","job_status":"start","queue_duration_s":0.51827,"scheduling_latency_s":0.518286} gitlab # [ 257.087678] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5fbfe04fc769b2a1b728e42","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5fbfe04fc769b2a1b728e42: done: 0.013909 sec","job_status":"done","queue_duration_s":0.51827,"scheduling_latency_s":0.518286,"redis_calls":13,"redis_duration_s":0.001346,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013909,"completed_at":"2026-09-05T11:58:03.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.126134] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f85fab9f328d0562e161f93c","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f85fab9f328d0562e161f93c: start","job_status":"start","queue_duration_s":0.533233,"scheduling_latency_s":0.533249} gitlab # [ 257.138484] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f85fab9f328d0562e161f93c","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f85fab9f328d0562e161f93c: done: 0.0134 sec","job_status":"done","queue_duration_s":0.533233,"scheduling_latency_s":0.533249,"redis_calls":13,"redis_duration_s":0.001334,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000846,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0134,"completed_at":"2026-09-05T11:58:03.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.177061] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9238369d11384f4d50da4bf","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9238369d11384f4d50da4bf: start","job_status":"start","queue_duration_s":0.547663,"scheduling_latency_s":0.547679} gitlab # [ 257.189464] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9238369d11384f4d50da4bf","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9238369d11384f4d50da4bf: done: 0.013713 sec","job_status":"done","queue_duration_s":0.547663,"scheduling_latency_s":0.547679,"redis_calls":13,"redis_duration_s":0.001417,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013713,"completed_at":"2026-09-05T11:58:03.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.229251] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5801d2144498b1658837e79c","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5801d2144498b1658837e79c: start","job_status":"start","queue_duration_s":0.56245,"scheduling_latency_s":0.562467} gitlab # [ 257.241369] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5801d2144498b1658837e79c","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5801d2144498b1658837e79c: done: 0.013448 sec","job_status":"done","queue_duration_s":0.56245,"scheduling_latency_s":0.562467,"redis_calls":13,"redis_duration_s":0.00132,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000839,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013448,"completed_at":"2026-09-05T11:58:03.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.280270] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12112a55d0885bf51c9236b4","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12112a55d0885bf51c9236b4: start","job_status":"start","queue_duration_s":0.576932,"scheduling_latency_s":0.576949} gitlab # [ 257.292269] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12112a55d0885bf51c9236b4","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12112a55d0885bf51c9236b4: done: 0.013381 sec","job_status":"done","queue_duration_s":0.576932,"scheduling_latency_s":0.576949,"redis_calls":13,"redis_duration_s":0.0013579999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013381,"completed_at":"2026-09-05T11:58:03.312Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.331127] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9181fc2d4804161d40a77ac9","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9181fc2d4804161d40a77ac9: start","job_status":"start","queue_duration_s":0.591403,"scheduling_latency_s":0.591419} gitlab # [ 257.343498] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9181fc2d4804161d40a77ac9","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9181fc2d4804161d40a77ac9: done: 0.013664 sec","job_status":"done","queue_duration_s":0.591403,"scheduling_latency_s":0.591419,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000844,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013664,"completed_at":"2026-09-05T11:58:03.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.382287] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0784289f0266b235f03a14b8","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0784289f0266b235f03a14b8: start","job_status":"start","queue_duration_s":0.606156,"scheduling_latency_s":0.606172} gitlab # [ 257.394727] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0784289f0266b235f03a14b8","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0784289f0266b235f03a14b8: done: 0.013404 sec","job_status":"done","queue_duration_s":0.606156,"scheduling_latency_s":0.606172,"redis_calls":13,"redis_duration_s":0.001324,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013404,"completed_at":"2026-09-05T11:58:03.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.435711] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c06c0e9e1054b6f33cecc6f6","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c06c0e9e1054b6f33cecc6f6: start","job_status":"start","queue_duration_s":0.620569,"scheduling_latency_s":0.620585} gitlab # [ 257.472725] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c06c0e9e1054b6f33cecc6f6","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c06c0e9e1054b6f33cecc6f6: done: 0.013761 sec","job_status":"done","queue_duration_s":0.620569,"scheduling_latency_s":0.620585,"redis_calls":13,"redis_duration_s":0.001351,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013761,"completed_at":"2026-09-05T11:58:03.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.543750] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19f5c5649f09cad615373b1f","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19f5c5649f09cad615373b1f: start","job_status":"start","queue_duration_s":0.63538,"scheduling_latency_s":0.635395} gitlab # [ 257.558218] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19f5c5649f09cad615373b1f","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19f5c5649f09cad615373b1f: done: 0.013686 sec","job_status":"done","queue_duration_s":0.63538,"scheduling_latency_s":0.635395,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013686,"completed_at":"2026-09-05T11:58:03.371Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.597578] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7166e1a1c2dc6190d6fc05b","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7166e1a1c2dc6190d6fc05b: start","job_status":"start","queue_duration_s":0.650176,"scheduling_latency_s":0.650191} gitlab # [ 257.609626] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7166e1a1c2dc6190d6fc05b","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7166e1a1c2dc6190d6fc05b: done: 0.013385 sec","job_status":"done","queue_duration_s":0.650176,"scheduling_latency_s":0.650191,"redis_calls":13,"redis_duration_s":0.0013319999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013385,"completed_at":"2026-09-05T11:58:03.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.649495] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2be90868a203b8264fe0f11a","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2be90868a203b8264fe0f11a: start","job_status":"start","queue_duration_s":0.664589,"scheduling_latency_s":0.664605} gitlab # [ 257.661729] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2be90868a203b8264fe0f11a","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2be90868a203b8264fe0f11a: done: 0.013469 sec","job_status":"done","queue_duration_s":0.664589,"scheduling_latency_s":0.664605,"redis_calls":13,"redis_duration_s":0.0013440000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013469,"completed_at":"2026-09-05T11:58:03.400Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.700145] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2717bdeac3f5172de76da005","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2717bdeac3f5172de76da005: start","job_status":"start","queue_duration_s":0.679137,"scheduling_latency_s":0.679153} gitlab # [ 257.712529] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2717bdeac3f5172de76da005","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2717bdeac3f5172de76da005: done: 0.013525 sec","job_status":"done","queue_duration_s":0.679137,"scheduling_latency_s":0.679153,"redis_calls":13,"redis_duration_s":0.001376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013525,"completed_at":"2026-09-05T11:58:03.415Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.751427] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25aca14b4cca7715bc7a4900","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25aca14b4cca7715bc7a4900: start","job_status":"start","queue_duration_s":0.69371,"scheduling_latency_s":0.693727} gitlab # [ 257.763783] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25aca14b4cca7715bc7a4900","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25aca14b4cca7715bc7a4900: done: 0.013454 sec","job_status":"done","queue_duration_s":0.69371,"scheduling_latency_s":0.693727,"redis_calls":13,"redis_duration_s":0.00134,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000851,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013454,"completed_at":"2026-09-05T11:58:03.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.802497] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"151c5fb4769f088b2e606f70","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-151c5fb4769f088b2e606f70: start","job_status":"start","queue_duration_s":0.70818,"scheduling_latency_s":0.708196} gitlab # [ 257.814995] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"151c5fb4769f088b2e606f70","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-151c5fb4769f088b2e606f70: done: 0.013694 sec","job_status":"done","queue_duration_s":0.70818,"scheduling_latency_s":0.708196,"redis_calls":13,"redis_duration_s":0.001354,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000841,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013694,"completed_at":"2026-09-05T11:58:03.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.854659] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"364e0467443ffb4a81f96f7f","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-364e0467443ffb4a81f96f7f: start","job_status":"start","queue_duration_s":0.723158,"scheduling_latency_s":0.723176} gitlab # [ 257.866754] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"364e0467443ffb4a81f96f7f","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-364e0467443ffb4a81f96f7f: done: 0.013559 sec","job_status":"done","queue_duration_s":0.723158,"scheduling_latency_s":0.723176,"redis_calls":13,"redis_duration_s":0.001363,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000864,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013559,"completed_at":"2026-09-05T11:58:03.459Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.904829] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae0565ef5894283bfe258af4","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae0565ef5894283bfe258af4: start","job_status":"start","queue_duration_s":0.737773,"scheduling_latency_s":0.73779} gitlab # [ 257.917009] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae0565ef5894283bfe258af4","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae0565ef5894283bfe258af4: done: 0.013336 sec","job_status":"done","queue_duration_s":0.737773,"scheduling_latency_s":0.73779,"redis_calls":13,"redis_duration_s":0.001347,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000848,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013336,"completed_at":"2026-09-05T11:58:03.473Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.954994] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc9d6a1f1099d6b352dc803a","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc9d6a1f1099d6b352dc803a: start","job_status":"start","queue_duration_s":0.752124,"scheduling_latency_s":0.752139} gitlab # [ 257.967156] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc9d6a1f1099d6b352dc803a","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc9d6a1f1099d6b352dc803a: done: 0.013661 sec","job_status":"done","queue_duration_s":0.752124,"scheduling_latency_s":0.752139,"redis_calls":13,"redis_duration_s":0.0013700000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000864,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013661,"completed_at":"2026-09-05T11:58:03.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.005391] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba6adaa0198fc95e3bae4487","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba6adaa0198fc95e3bae4487: start","job_status":"start","queue_duration_s":0.766814,"scheduling_latency_s":0.76683} gitlab # [ 258.017427] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba6adaa0198fc95e3bae4487","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba6adaa0198fc95e3bae4487: done: 0.013336 sec","job_status":"done","queue_duration_s":0.766814,"scheduling_latency_s":0.76683,"redis_calls":13,"redis_duration_s":0.001322,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013336,"completed_at":"2026-09-05T11:58:03.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.056475] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8386e543c6b3e27f12cbe62","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8386e543c6b3e27f12cbe62: start","job_status":"start","queue_duration_s":0.781218,"scheduling_latency_s":0.781234} gitlab # [ 258.068831] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8386e543c6b3e27f12cbe62","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8386e543c6b3e27f12cbe62: done: 0.013417 sec","job_status":"done","queue_duration_s":0.781218,"scheduling_latency_s":0.781234,"redis_calls":13,"redis_duration_s":0.00136,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000878,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013417,"completed_at":"2026-09-05T11:58:03.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.107462] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a79646d4fa51f5e8000f95df","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a79646d4fa51f5e8000f95df: start","job_status":"start","queue_duration_s":0.795654,"scheduling_latency_s":0.795671} gitlab # [ 258.119797] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a79646d4fa51f5e8000f95df","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a79646d4fa51f5e8000f95df: done: 0.013615 sec","job_status":"done","queue_duration_s":0.795654,"scheduling_latency_s":0.795671,"redis_calls":13,"redis_duration_s":0.001362,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013615,"completed_at":"2026-09-05T11:58:03.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.158326] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c3e7b0e1247b1a64320259","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c3e7b0e1247b1a64320259: start","job_status":"start","queue_duration_s":0.810288,"scheduling_latency_s":0.810303} gitlab # [ 258.170604] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c3e7b0e1247b1a64320259","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c3e7b0e1247b1a64320259: done: 0.013579 sec","job_status":"done","queue_duration_s":0.810288,"scheduling_latency_s":0.810303,"redis_calls":13,"redis_duration_s":0.001335,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013579,"completed_at":"2026-09-05T11:58:03.546Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.209117] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d653f8d666fd5e3261285415","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d653f8d666fd5e3261285415: start","job_status":"start","queue_duration_s":0.82495,"scheduling_latency_s":0.824966} gitlab # [ 258.221300] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d653f8d666fd5e3261285415","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d653f8d666fd5e3261285415: done: 0.013483 sec","job_status":"done","queue_duration_s":0.82495,"scheduling_latency_s":0.824966,"redis_calls":13,"redis_duration_s":0.001369,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013483,"completed_at":"2026-09-05T11:58:03.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.260926] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b6722efda918c0efa6d5d6c","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b6722efda918c0efa6d5d6c: start","job_status":"start","queue_duration_s":0.839475,"scheduling_latency_s":0.839491} gitlab # [ 258.273273] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b6722efda918c0efa6d5d6c","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b6722efda918c0efa6d5d6c: done: 0.01362 sec","job_status":"done","queue_duration_s":0.839475,"scheduling_latency_s":0.839491,"redis_calls":13,"redis_duration_s":0.0013549999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01362,"completed_at":"2026-09-05T11:58:03.575Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.311993] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e413dae52859957ebf8f9bec","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e413dae52859957ebf8f9bec: start","job_status":"start","queue_duration_s":0.85413,"scheduling_latency_s":0.854146} gitlab # [ 258.323854] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e413dae52859957ebf8f9bec","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e413dae52859957ebf8f9bec: done: 0.013355 sec","job_status":"done","queue_duration_s":0.85413,"scheduling_latency_s":0.854146,"redis_calls":13,"redis_duration_s":0.001324,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013355,"completed_at":"2026-09-05T11:58:03.589Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.362948] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e31db36687a38989fc919207","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e31db36687a38989fc919207: start","job_status":"start","queue_duration_s":0.868517,"scheduling_latency_s":0.868533} gitlab # [ 258.375199] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e31db36687a38989fc919207","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e31db36687a38989fc919207: done: 0.013462 sec","job_status":"done","queue_duration_s":0.868517,"scheduling_latency_s":0.868533,"redis_calls":13,"redis_duration_s":0.00133,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013462,"completed_at":"2026-09-05T11:58:03.604Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.412936] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b92224f8d888f34be22169ba","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b92224f8d888f34be22169ba: start","job_status":"start","queue_duration_s":0.882985,"scheduling_latency_s":0.883} gitlab # [ 258.425554] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b92224f8d888f34be22169ba","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b92224f8d888f34be22169ba: done: 0.013511 sec","job_status":"done","queue_duration_s":0.882985,"scheduling_latency_s":0.883,"redis_calls":13,"redis_duration_s":0.001366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013511,"completed_at":"2026-09-05T11:58:03.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.465432] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bd7c1f5ee1ab36cab2746c5","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bd7c1f5ee1ab36cab2746c5: start","job_status":"start","queue_duration_s":0.897624,"scheduling_latency_s":0.89764} gitlab # [ 258.477561] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bd7c1f5ee1ab36cab2746c5","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bd7c1f5ee1ab36cab2746c5: done: 0.013297 sec","job_status":"done","queue_duration_s":0.897624,"scheduling_latency_s":0.89764,"redis_calls":13,"redis_duration_s":0.001329,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013297,"completed_at":"2026-09-05T11:58:03.633Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.516548] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"328f2a785de062db24a5ea88","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-328f2a785de062db24a5ea88: start","job_status":"start","queue_duration_s":0.911945,"scheduling_latency_s":0.911962} gitlab # [ 258.528838] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"328f2a785de062db24a5ea88","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-328f2a785de062db24a5ea88: done: 0.013677 sec","job_status":"done","queue_duration_s":0.911945,"scheduling_latency_s":0.911962,"redis_calls":13,"redis_duration_s":0.0013180000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013677,"completed_at":"2026-09-05T11:58:03.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.568503] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e3c7f5b0417022efd49f981","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e3c7f5b0417022efd49f981: start","job_status":"start","queue_duration_s":0.926654,"scheduling_latency_s":0.92667} gitlab # [ 258.580910] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e3c7f5b0417022efd49f981","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e3c7f5b0417022efd49f981: done: 0.013634 sec","job_status":"done","queue_duration_s":0.926654,"scheduling_latency_s":0.92667,"redis_calls":13,"redis_duration_s":0.0013709999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013634,"completed_at":"2026-09-05T11:58:03.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.620135] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b243f03dac6e39512f363f20","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b243f03dac6e39512f363f20: start","job_status":"start","queue_duration_s":0.941386,"scheduling_latency_s":0.941402} gitlab # [ 258.632294] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b243f03dac6e39512f363f20","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b243f03dac6e39512f363f20: done: 0.013577 sec","job_status":"done","queue_duration_s":0.941386,"scheduling_latency_s":0.941402,"redis_calls":13,"redis_duration_s":0.001354,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000859,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013577,"completed_at":"2026-09-05T11:58:03.677Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.672313] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fbfc45fb56c1cfe0b9e809e","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fbfc45fb56c1cfe0b9e809e: start","job_status":"start","queue_duration_s":0.956066,"scheduling_latency_s":0.956083} gitlab # [ 258.684636] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fbfc45fb56c1cfe0b9e809e","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fbfc45fb56c1cfe0b9e809e: done: 0.01347 sec","job_status":"done","queue_duration_s":0.956066,"scheduling_latency_s":0.956083,"redis_calls":13,"redis_duration_s":0.001343,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01347,"completed_at":"2026-09-05T11:58:03.691Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.723557] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5abd65514294cc4db051e8c8","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5abd65514294cc4db051e8c8: start","job_status":"start","queue_duration_s":0.970535,"scheduling_latency_s":0.970552} gitlab # [ 258.736063] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5abd65514294cc4db051e8c8","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5abd65514294cc4db051e8c8: done: 0.014162 sec","job_status":"done","queue_duration_s":0.970535,"scheduling_latency_s":0.970552,"redis_calls":13,"redis_duration_s":0.001433,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014162,"completed_at":"2026-09-05T11:58:03.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.775116] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1936d809c2751c09f47a4712","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1936d809c2751c09f47a4712: start","job_status":"start","queue_duration_s":0.985811,"scheduling_latency_s":0.985829} gitlab # [ 258.787778] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1936d809c2751c09f47a4712","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1936d809c2751c09f47a4712: done: 0.013543 sec","job_status":"done","queue_duration_s":0.985811,"scheduling_latency_s":0.985829,"redis_calls":13,"redis_duration_s":0.0014169999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013543,"completed_at":"2026-09-05T11:58:03.721Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.827276] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9bed45384145bd188b0bc95","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9bed45384145bd188b0bc95: start","job_status":"start","queue_duration_s":1.000366,"scheduling_latency_s":1.000381} gitlab # [ 258.839821] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9bed45384145bd188b0bc95","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9bed45384145bd188b0bc95: done: 0.013432 sec","job_status":"done","queue_duration_s":1.000366,"scheduling_latency_s":1.000381,"redis_calls":13,"redis_duration_s":0.001355,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013432,"completed_at":"2026-09-05T11:58:03.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.879220] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"052db3440f8408d759667fc3","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-052db3440f8408d759667fc3: start","job_status":"start","queue_duration_s":1.014837,"scheduling_latency_s":1.014853} gitlab # [ 258.891540] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"052db3440f8408d759667fc3","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-052db3440f8408d759667fc3: done: 0.013793 sec","job_status":"done","queue_duration_s":1.014837,"scheduling_latency_s":1.014853,"redis_calls":13,"redis_duration_s":0.001583,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000876,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013793,"completed_at":"2026-09-05T11:58:03.751Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.930110] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"660a71aee9f0c538a8f26412","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-660a71aee9f0c538a8f26412: start","job_status":"start","queue_duration_s":1.029742,"scheduling_latency_s":1.029759} gitlab # [ 258.942314] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"660a71aee9f0c538a8f26412","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-660a71aee9f0c538a8f26412: done: 0.01337 sec","job_status":"done","queue_duration_s":1.029742,"scheduling_latency_s":1.029759,"redis_calls":13,"redis_duration_s":0.0013540000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000848,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01337,"completed_at":"2026-09-05T11:58:03.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.980827] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed5b79f7b8a016f72401a210","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed5b79f7b8a016f72401a210: start","job_status":"start","queue_duration_s":1.044124,"scheduling_latency_s":1.04414} gitlab # [ 258.992733] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed5b79f7b8a016f72401a210","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed5b79f7b8a016f72401a210: done: 0.013453 sec","job_status":"done","queue_duration_s":1.044124,"scheduling_latency_s":1.04414,"redis_calls":13,"redis_duration_s":0.001363,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000842,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013453,"completed_at":"2026-09-05T11:58:03.780Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.031100] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5fa822e65e4af906fedd574","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5fa822e65e4af906fedd574: start","job_status":"start","queue_duration_s":1.05862,"scheduling_latency_s":1.058636} gitlab # [ 259.043516] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5fa822e65e4af906fedd574","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5fa822e65e4af906fedd574: done: 0.013593 sec","job_status":"done","queue_duration_s":1.05862,"scheduling_latency_s":1.058636,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013593,"completed_at":"2026-09-05T11:58:03.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.082910] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df1de2226051b821c473cbbf","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df1de2226051b821c473cbbf: start","job_status":"start","queue_duration_s":1.073292,"scheduling_latency_s":1.073307} gitlab # [ 259.095286] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df1de2226051b821c473cbbf","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df1de2226051b821c473cbbf: done: 0.013301 sec","job_status":"done","queue_duration_s":1.073292,"scheduling_latency_s":1.073307,"redis_calls":13,"redis_duration_s":0.001328,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013301,"completed_at":"2026-09-05T11:58:03.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.134143] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ac4178c251a31b25960feb9","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ac4178c251a31b25960feb9: start","job_status":"start","queue_duration_s":1.08762,"scheduling_latency_s":1.087636} gitlab # [ 259.146322] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ac4178c251a31b25960feb9","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ac4178c251a31b25960feb9: done: 0.01339 sec","job_status":"done","queue_duration_s":1.08762,"scheduling_latency_s":1.087636,"redis_calls":13,"redis_duration_s":0.001332,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01339,"completed_at":"2026-09-05T11:58:03.823Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.183915] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea05f1f909bd07ae122f9533","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea05f1f909bd07ae122f9533: start","job_status":"start","queue_duration_s":1.102108,"scheduling_latency_s":1.102124} gitlab # [ 259.195958] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea05f1f909bd07ae122f9533","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea05f1f909bd07ae122f9533: done: 0.013604 sec","job_status":"done","queue_duration_s":1.102108,"scheduling_latency_s":1.102124,"redis_calls":13,"redis_duration_s":0.0013570000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013604,"completed_at":"2026-09-05T11:58:03.838Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.234819] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68f112644be14401745585e3","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68f112644be14401745585e3: start","job_status":"start","queue_duration_s":1.116726,"scheduling_latency_s":1.116741} gitlab # [ 259.246941] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68f112644be14401745585e3","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68f112644be14401745585e3: done: 0.013615 sec","job_status":"done","queue_duration_s":1.116726,"scheduling_latency_s":1.116741,"redis_calls":13,"redis_duration_s":0.0015110000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013615,"completed_at":"2026-09-05T11:58:03.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.286591] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a40b6a46d36c38b3b62f567","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a40b6a46d36c38b3b62f567: start","job_status":"start","queue_duration_s":1.131353,"scheduling_latency_s":1.131369} gitlab # [ 259.298853] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a40b6a46d36c38b3b62f567","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a40b6a46d36c38b3b62f567: done: 0.013933 sec","job_status":"done","queue_duration_s":1.131353,"scheduling_latency_s":1.131369,"redis_calls":13,"redis_duration_s":0.001349,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013933,"completed_at":"2026-09-05T11:58:03.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.337793] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"260c4ec84bee96b3774f5712","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-260c4ec84bee96b3774f5712: start","job_status":"start","queue_duration_s":1.146317,"scheduling_latency_s":1.146334} gitlab # [ 259.349880] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"260c4ec84bee96b3774f5712","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-260c4ec84bee96b3774f5712: done: 0.013579 sec","job_status":"done","queue_duration_s":1.146317,"scheduling_latency_s":1.146334,"redis_calls":13,"redis_duration_s":0.001374,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000878,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013579,"completed_at":"2026-09-05T11:58:03.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.388103] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89e4e10e604db7eea23f1512","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89e4e10e604db7eea23f1512: start","job_status":"start","queue_duration_s":1.160963,"scheduling_latency_s":1.16098} gitlab # [ 259.399960] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:58:03.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89e4e10e604db7eea23f1512","created_at":"2026-09-05T11:58:02.650Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-bb3801ac641b453f","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8afbfc933ce6d4916c58512a1cca4666","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:58:02.722Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89e4e10e604db7eea23f1512: done: 0.014593 sec","job_status":"done","queue_duration_s":1.160963,"scheduling_latency_s":1.16098,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014593,"completed_at":"2026-09-05T11:58:03.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.960845] setup-gitlab-start[2981]: W, [2026-09-05T11:58:22.843000 #2981] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 308.186556] redis[788]: 10 changes in 300 seconds. Saving... gitlab # [ 308.192274] redis[788]: Background saving started by pid 3299 gitlab # [ 308.196330] redis[3299]: BGSAVE done, 443 keys saved, 0 keys skipped, 115797 bytes written. gitlab # [ 308.203225] redis[3299]: DB saved on disk gitlab # [ 308.205571] redis[3299]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB gitlab # [ 308.287735] redis[788]: Background saving terminated with success gitlab # [ 309.806013] postgres[870]: [870] LOG: checkpoint starting: time gitlab # [ 311.130653] gitaly[897]: time="2026-09-05T11:58:58.012Z" level=info msg="burden monitor stats" pid=897 gitlab # [ 315.068554] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:01.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"63527e1fbfde9298f29a7723","created_at":"2026-09-05T11:59:01.883Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6b08c8967582be05f00405f7608e41c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:01.947Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-63527e1fbfde9298f29a7723: start","job_status":"start","queue_duration_s":0.002929,"scheduling_latency_s":0.002971} gitlab # [ 315.226603] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.106Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"63527e1fbfde9298f29a7723","created_at":"2026-09-05T11:59:01.883Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6b08c8967582be05f00405f7608e41c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:01.947Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-63527e1fbfde9298f29a7723: done: 0.156356 sec","job_status":"done","queue_duration_s":0.002929,"scheduling_latency_s":0.002971,"redis_calls":16,"redis_duration_s":0.042515000000000004,"redis_read_bytes":1157,"redis_write_bytes":1414,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.032313,"redis_feature_flag_read_bytes":1152,"redis_feature_flag_write_bytes":638,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007643,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002559,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.156356,"completed_at":"2026-09-05T11:59:02.106Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 315.309118] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"741c3af33491c3e074c77281","created_at":"2026-09-05T11:59:01.975Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9523eb91b4325b0c00ae8b73d03a3db8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:01.982Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-741c3af33491c3e074c77281: start","job_status":"start","queue_duration_s":0.160663,"scheduling_latency_s":0.160711} gitlab # [ 315.337777] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.220Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 315.371698] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.253Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"741c3af33491c3e074c77281","created_at":"2026-09-05T11:59:01.975Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9523eb91b4325b0c00ae8b73d03a3db8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:01.982Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-741c3af33491c3e074c77281: done: 0.111099 sec","job_status":"done","queue_duration_s":0.160663,"scheduling_latency_s":0.160711,"redis_calls":8,"redis_duration_s":0.011996,"redis_read_bytes":133,"redis_write_bytes":931,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.002031,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007336,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002629,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.111099,"completed_at":"2026-09-05T11:59:02.253Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 315.445331] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2488bca21000ee9750f264ef","created_at":"2026-09-05T11:59:01.986Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3dd96c7accbaa3e181259e1a6c2788c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.004Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2488bca21000ee9750f264ef: start","job_status":"start","queue_duration_s":0.253878,"scheduling_latency_s":0.253927} gitlab # [ 315.461110] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.298Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 315.488090] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2488bca21000ee9750f264ef","created_at":"2026-09-05T11:59:01.986Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3dd96c7accbaa3e181259e1a6c2788c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.004Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2488bca21000ee9750f264ef: done: 0.112631 sec","job_status":"done","queue_duration_s":0.253878,"scheduling_latency_s":0.253927,"redis_calls":8,"redis_duration_s":0.00906,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":101,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004586,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004177,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.112631,"completed_at":"2026-09-05T11:59:02.371Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 315.538925] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"97bd5af26bf318938cb67e9e","created_at":"2026-09-05T11:59:02.043Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f91600d7babbc2a5e5be6c97ff7492ca","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.056Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-97bd5af26bf318938cb67e9e: start","job_status":"start","queue_duration_s":0.321722,"scheduling_latency_s":0.321766} gitlab # [ 315.809588] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.692Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"97bd5af26bf318938cb67e9e","created_at":"2026-09-05T11:59:02.043Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f91600d7babbc2a5e5be6c97ff7492ca","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.056Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-97bd5af26bf318938cb67e9e: done: 0.313446 sec","job_status":"done","queue_duration_s":0.321722,"scheduling_latency_s":0.321766,"redis_calls":10,"redis_duration_s":0.016885999999999998,"redis_read_bytes":135,"redis_write_bytes":1275,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001204,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":106,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009283,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0015992,"exclusive_lock_hold_duration_s":0.05230742,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.313446,"completed_at":"2026-09-05T11:59:02.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.163621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 315.880521] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"4c69b139d72a870ca5acf5fb","created_at":"2026-09-05T11:59:02.088Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fdded053c15865ea72894ac84929e05c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.096Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-4c69b139d72a870ca5acf5fb: start","job_status":"start","queue_duration_s":0.599013,"scheduling_latency_s":0.599053} gitlab # [ 315.905035] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"4c69b139d72a870ca5acf5fb","created_at":"2026-09-05T11:59:02.088Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fdded053c15865ea72894ac84929e05c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.096Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-4c69b139d72a870ca5acf5fb: done: 0.029412 sec","job_status":"done","queue_duration_s":0.599013,"scheduling_latency_s":0.599053,"redis_calls":9,"redis_duration_s":0.002726,"redis_read_bytes":261,"redis_write_bytes":813,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000515,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":132,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029412,"completed_at":"2026-09-05T11:59:02.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 315.954287] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:02.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d3fec07f5d41846237826b5d","created_at":"2026-09-05T11:59:02.137Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ac82b1768979b19ee3028c4db4762f9","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.148Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d3fec07f5d41846237826b5d: start","job_status":"start","queue_duration_s":0.578184,"scheduling_latency_s":0.578213} gitlab # [ 316.162493] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.044Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d3fec07f5d41846237826b5d","created_at":"2026-09-05T11:59:02.137Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ac82b1768979b19ee3028c4db4762f9","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.148Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d3fec07f5d41846237826b5d: done: 0.318366 sec","job_status":"done","queue_duration_s":0.578184,"scheduling_latency_s":0.578213,"redis_calls":563,"redis_duration_s":0.07007000000000001,"redis_read_bytes":688,"redis_write_bytes":48843,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000151,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.069011,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.282809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.318366,"completed_at":"2026-09-05T11:59:03.044Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.213205] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.046Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"be8b1a7a723e6e37ded65e3e","created_at":"2026-09-05T11:59:02.164Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dacf1a041e7c764f05dc59b0de48d1b3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.171Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-be8b1a7a723e6e37ded65e3e: start","job_status":"start","queue_duration_s":0.875428,"scheduling_latency_s":0.875446} gitlab # [ 316.227360] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"be8b1a7a723e6e37ded65e3e","created_at":"2026-09-05T11:59:02.164Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dacf1a041e7c764f05dc59b0de48d1b3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.171Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-be8b1a7a723e6e37ded65e3e: done: 0.025872 sec","job_status":"done","queue_duration_s":0.875428,"scheduling_latency_s":0.875446,"redis_calls":10,"redis_duration_s":0.0029620000000000002,"redis_read_bytes":135,"redis_write_bytes":1184,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000209,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":97,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001308,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005275,"exclusive_lock_hold_duration_s":0.00343344,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.025872,"completed_at":"2026-09-05T11:59:03.072Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.270853] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.075Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"670cda825f9a5a08498ef6c3","created_at":"2026-09-05T11:59:02.196Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfab5ef855cfca0c13ed60664203b063","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.218Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-670cda825f9a5a08498ef6c3: start","job_status":"start","queue_duration_s":0.857359,"scheduling_latency_s":0.857403} gitlab # [ 316.285182] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"670cda825f9a5a08498ef6c3","created_at":"2026-09-05T11:59:02.196Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfab5ef855cfca0c13ed60664203b063","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.218Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-670cda825f9a5a08498ef6c3: done: 0.047349 sec","job_status":"done","queue_duration_s":0.857359,"scheduling_latency_s":0.857403,"redis_calls":11,"redis_duration_s":0.003584,"redis_read_bytes":517,"redis_write_bytes":1011,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001344,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":300,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000645,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047349,"completed_at":"2026-09-05T11:59:03.122Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.326352] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.125Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"225546077e1e57f15e05013c","created_at":"2026-09-05T11:59:02.238Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d1802d3fe748613e149a23745ee1e61","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.249Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-225546077e1e57f15e05013c: start","job_status":"start","queue_duration_s":0.876358,"scheduling_latency_s":0.876395} gitlab # [ 316.339623] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"225546077e1e57f15e05013c","created_at":"2026-09-05T11:59:02.238Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d1802d3fe748613e149a23745ee1e61","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.249Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-225546077e1e57f15e05013c: done: 0.033315 sec","job_status":"done","queue_duration_s":0.876358,"scheduling_latency_s":0.876395,"redis_calls":10,"redis_duration_s":0.0038510000000000003,"redis_read_bytes":135,"redis_write_bytes":1167,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000304,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00088564,"exclusive_lock_hold_duration_s":0.00505954,"duration_s":0.033315,"completed_at":"2026-09-05T11:59:03.158Z","load_balancing_strategy":"primary","db_duration_s":0.001189,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.380974] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.161Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"d0117876214248fdbbfbd485","created_at":"2026-09-05T11:59:02.265Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0de054e394f781d72cf58828763c569e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-d0117876214248fdbbfbd485: start","job_status":"start","queue_duration_s":0.875639,"scheduling_latency_s":0.87567} gitlab # [ 316.394969] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.196Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"d0117876214248fdbbfbd485","created_at":"2026-09-05T11:59:02.265Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0de054e394f781d72cf58828763c569e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.285Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-d0117876214248fdbbfbd485: done: 0.035232 sec","job_status":"done","queue_duration_s":0.875639,"scheduling_latency_s":0.87567,"redis_calls":11,"redis_duration_s":0.00313,"redis_read_bytes":136,"redis_write_bytes":1339,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000232,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":100,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001609,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001289,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00759664,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.035232,"completed_at":"2026-09-05T11:59:03.196Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003431,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 316.439347] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"4568aee3ba6bd7bf8dbd3156","created_at":"2026-09-05T11:59:02.363Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01d605bff4e5259378a5d07d5c205461","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.373Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-4568aee3ba6bd7bf8dbd3156: start","job_status":"start","queue_duration_s":0.824521,"scheduling_latency_s":0.82455} gitlab # [ 316.453740] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.210Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 316.456891] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"4568aee3ba6bd7bf8dbd3156","created_at":"2026-09-05T11:59:02.363Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01d605bff4e5259378a5d07d5c205461","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.373Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-4568aee3ba6bd7bf8dbd3156: done: 0.019318 sec","job_status":"done","queue_duration_s":0.824521,"scheduling_latency_s":0.82455,"redis_calls":8,"redis_duration_s":0.0015179999999999998,"redis_read_bytes":133,"redis_write_bytes":873,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000168,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":102,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019318,"completed_at":"2026-09-05T11:59:03.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.498606] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"caf523f0ae4f7442e8a187a3","created_at":"2026-09-05T11:59:02.446Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef5138fbf5d3946b142dfcfddad6dd60","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.453Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-caf523f0ae4f7442e8a187a3: start","job_status":"start","queue_duration_s":0.765533,"scheduling_latency_s":0.765558} gitlab # [ 316.512056] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"caf523f0ae4f7442e8a187a3","created_at":"2026-09-05T11:59:02.446Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef5138fbf5d3946b142dfcfddad6dd60","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.453Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-caf523f0ae4f7442e8a187a3: done: 0.033333 sec","job_status":"done","queue_duration_s":0.765533,"scheduling_latency_s":0.765558,"redis_calls":8,"redis_duration_s":0.0019230000000000002,"redis_read_bytes":133,"redis_write_bytes":936,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000161,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.033333,"completed_at":"2026-09-05T11:59:03.252Z","load_balancing_strategy":"primary","db_duration_s":0.012575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.554199] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c0fad8845d11ef2baadafb18","created_at":"2026-09-05T11:59:02.475Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.482Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c0fad8845d11ef2baadafb18: start","job_status":"start","queue_duration_s":0.77225,"scheduling_latency_s":0.772277} gitlab # [ 316.567349] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c0fad8845d11ef2baadafb18","created_at":"2026-09-05T11:59:02.475Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.482Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c0fad8845d11ef2baadafb18: done: 0.036068 sec","job_status":"done","queue_duration_s":0.77225,"scheduling_latency_s":0.772277,"redis_calls":16,"redis_duration_s":0.00294,"redis_read_bytes":272,"redis_write_bytes":4229,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000362,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":199,"redis_queues_calls":2,"redis_queues_duration_s":0.00033,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001691,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000557,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036068,"completed_at":"2026-09-05T11:59:03.290Z","load_balancing_strategy":"primary","db_duration_s":0.001877,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.608588] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.291Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"25d2716cc026bdff4153955b","created_at":"2026-09-05T11:59:02.494Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"385498ea8dca3e0ed49946c90e07c724","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.498Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-25d2716cc026bdff4153955b: start","job_status":"start","queue_duration_s":0.792962,"scheduling_latency_s":0.792986} gitlab # [ 316.623617] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"25d2716cc026bdff4153955b","created_at":"2026-09-05T11:59:02.494Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"385498ea8dca3e0ed49946c90e07c724","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.498Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-25d2716cc026bdff4153955b: done: 0.064883 sec","job_status":"done","queue_duration_s":0.792962,"scheduling_latency_s":0.792986,"redis_calls":11,"redis_duration_s":0.001739,"redis_read_bytes":263,"redis_write_bytes":1334,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":172,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0007,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051048,"exclusive_lock_hold_duration_s":0.0479615,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.064883,"completed_at":"2026-09-05T11:59:03.356Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039499,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.667074] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3f8b6be8e44c054832dd1ca1","created_at":"2026-09-05T11:59:02.534Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"925c04fb1f34ea36f64baf087cd27ae7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-05T11:59:02.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3f8b6be8e44c054832dd1ca1: start","job_status":"start","queue_duration_s":0.813072,"scheduling_latency_s":0.813091} gitlab # [ 316.679961] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3f8b6be8e44c054832dd1ca1","created_at":"2026-09-05T11:59:02.534Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"925c04fb1f34ea36f64baf087cd27ae7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-05T11:59:02.545Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3f8b6be8e44c054832dd1ca1: done: 0.022799 sec","job_status":"done","queue_duration_s":0.813072,"scheduling_latency_s":0.813091,"redis_calls":8,"redis_duration_s":0.001126,"redis_read_bytes":133,"redis_write_bytes":836,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00012,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022799,"completed_at":"2026-09-05T11:59:03.380Z","load_balancing_strategy":"primary","db_duration_s":0.002884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.719895] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d48ed0ce906b1c32d7403e2e","created_at":"2026-09-05T11:59:02.554Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67d31f730af806b90bbb3f25c845384e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-05T11:59:02.557Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d48ed0ce906b1c32d7403e2e: start","job_status":"start","queue_duration_s":0.824691,"scheduling_latency_s":0.824708} gitlab # [ 316.733191] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.389Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 316.736792] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d48ed0ce906b1c32d7403e2e","created_at":"2026-09-05T11:59:02.554Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67d31f730af806b90bbb3f25c845384e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-05T11:59:02.557Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d48ed0ce906b1c32d7403e2e: done: 0.012979 sec","job_status":"done","queue_duration_s":0.824691,"scheduling_latency_s":0.824708,"redis_calls":8,"redis_duration_s":0.001071,"redis_read_bytes":133,"redis_write_bytes":908,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000114,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":107,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012979,"completed_at":"2026-09-05T11:59:03.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.777341] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b2356b5848f23e86c96e503f","created_at":"2026-09-05T11:59:02.561Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0047cb8387a4c9ee00d74b091ef2d384","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.570Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b2356b5848f23e86c96e503f: start","job_status":"start","queue_duration_s":0.825905,"scheduling_latency_s":0.825923} gitlab # [ 316.791699] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.409Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b2356b5848f23e86c96e503f","created_at":"2026-09-05T11:59:02.561Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0047cb8387a4c9ee00d74b091ef2d384","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.570Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b2356b5848f23e86c96e503f: done: 0.013141 sec","job_status":"done","queue_duration_s":0.825905,"scheduling_latency_s":0.825923,"redis_calls":8,"redis_duration_s":0.0010400000000000001,"redis_read_bytes":133,"redis_write_bytes":943,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000114,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":112,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013141,"completed_at":"2026-09-05T11:59:03.409Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.833998] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"694df7de8cb0878715d371e9","created_at":"2026-09-05T11:59:02.580Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.585Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-694df7de8cb0878715d371e9: start","job_status":"start","queue_duration_s":0.824919,"scheduling_latency_s":0.824936} gitlab # [ 316.847273] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"694df7de8cb0878715d371e9","created_at":"2026-09-05T11:59:02.580Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.585Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-694df7de8cb0878715d371e9: done: 0.138083 sec","job_status":"done","queue_duration_s":0.824919,"scheduling_latency_s":0.824936,"redis_calls":65,"redis_duration_s":0.015851,"redis_read_bytes":748,"redis_write_bytes":41502,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.000545,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.000992,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013786,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.138083,"completed_at":"2026-09-05T11:59:03.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.889320] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.550Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"68c2dca806f5adab95301d9c","created_at":"2026-09-05T11:59:02.592Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86eeabcfa27a01e0a362942cff0c019c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.601Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-68c2dca806f5adab95301d9c: start","job_status":"start","queue_duration_s":0.949474,"scheduling_latency_s":0.949494} gitlab # [ 316.914243] postgres[3317]: [3317] ERROR: relation "table_without_models" does not exist gitlab # [ 316.915382] postgres[3317]: [3317] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 316.916708] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"68c2dca806f5adab95301d9c","created_at":"2026-09-05T11:59:02.592Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86eeabcfa27a01e0a362942cff0c019c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.601Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-68c2dca806f5adab95301d9c: done: 0.24854 sec","job_status":"done","queue_duration_s":0.949474,"scheduling_latency_s":0.949494,"redis_calls":10,"redis_duration_s":0.002071,"redis_read_bytes":135,"redis_write_bytes":1071,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000219,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":84,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000808,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.042,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.19902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00054148,"exclusive_lock_hold_duration_s":0.23281508,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.24854,"completed_at":"2026-09-05T11:59:03.799Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.060071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 316.960263] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"770634064aeb52dbec16ee54","created_at":"2026-09-05T11:59:02.611Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5482d46deae72c32cb89705b633fa633","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.618Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-770634064aeb52dbec16ee54: start","job_status":"start","queue_duration_s":1.182353,"scheduling_latency_s":1.182372} gitlab # [ 316.974498] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.808Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 316.977883] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"770634064aeb52dbec16ee54","created_at":"2026-09-05T11:59:02.611Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5482d46deae72c32cb89705b633fa633","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:02.618Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-770634064aeb52dbec16ee54: done: 0.012978 sec","job_status":"done","queue_duration_s":1.182353,"scheduling_latency_s":1.182372,"redis_calls":8,"redis_duration_s":0.001163,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000117,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012978,"completed_at":"2026-09-05T11:59:03.813Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.019712] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0601eba59ad3d69d1748c79b","created_at":"2026-09-05T11:59:02.648Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68dbdbf4028caab930d3732db26cec1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-05T11:59:02.657Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0601eba59ad3d69d1748c79b: start","job_status":"start","queue_duration_s":1.156896,"scheduling_latency_s":1.156914} gitlab # [ 317.034261] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.822Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 317.037596] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.827Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0601eba59ad3d69d1748c79b","created_at":"2026-09-05T11:59:02.648Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-2923e5491048476e","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68dbdbf4028caab930d3732db26cec1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-05T11:59:02.657Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0601eba59ad3d69d1748c79b: done: 0.013051 sec","job_status":"done","queue_duration_s":1.156896,"scheduling_latency_s":1.156914,"redis_calls":8,"redis_duration_s":0.001095,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00012,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013051,"completed_at":"2026-09-05T11:59:03.827Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.078344] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.829Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5d91780e1942bf5f82f3447d","created_at":"2026-09-05T11:59:03.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-545e6ff6f9174291","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.283Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5d91780e1942bf5f82f3447d: start","job_status":"start","queue_duration_s":0.545472,"scheduling_latency_s":0.545492} gitlab # [ 317.090678] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.840Z","message":"No existing merge request to be cleaned up."} gitlab # [ 317.092344] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.848Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5d91780e1942bf5f82f3447d","created_at":"2026-09-05T11:59:03.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-545e6ff6f9174291","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.283Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5d91780e1942bf5f82f3447d: done: 0.019149 sec","job_status":"done","queue_duration_s":0.545472,"scheduling_latency_s":0.545492,"redis_calls":11,"redis_duration_s":0.001424,"redis_read_bytes":136,"redis_write_bytes":1104,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000114,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":82,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000758,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019149,"completed_at":"2026-09-05T11:59:03.848Z","load_balancing_strategy":"primary","db_duration_s":0.001844,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.132333] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.849Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"83ae7c5de27ba891e8dbfd8b","created_at":"2026-09-05T11:59:03.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-545e6ff6f9174291","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.283Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-83ae7c5de27ba891e8dbfd8b: start","job_status":"start","queue_duration_s":0.565679,"scheduling_latency_s":0.565695} gitlab # [ 317.144426] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.859Z","message":"No existing merge request to be cleaned up."} gitlab # [ 317.146072] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.867Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"83ae7c5de27ba891e8dbfd8b","created_at":"2026-09-05T11:59:03.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-545e6ff6f9174291","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.283Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-83ae7c5de27ba891e8dbfd8b: done: 0.018142 sec","job_status":"done","queue_duration_s":0.565679,"scheduling_latency_s":0.565695,"redis_calls":10,"redis_duration_s":0.0012759999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000725,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018142,"completed_at":"2026-09-05T11:59:03.867Z","load_balancing_strategy":"primary","db_duration_s":0.001367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.185144] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.868Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dca5acc1501b71fa2558b0b8","created_at":"2026-09-05T11:59:03.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-545e6ff6f9174291","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.283Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dca5acc1501b71fa2558b0b8: start","job_status":"start","queue_duration_s":0.584898,"scheduling_latency_s":0.584914} gitlab # [ 317.197336] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.878Z","message":"No existing merge request to be cleaned up."} gitlab # [ 317.198966] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.886Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dca5acc1501b71fa2558b0b8","created_at":"2026-09-05T11:59:03.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-545e6ff6f9174291","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.283Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dca5acc1501b71fa2558b0b8: done: 0.017955 sec","job_status":"done","queue_duration_s":0.584898,"scheduling_latency_s":0.584914,"redis_calls":10,"redis_duration_s":0.0013709999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000628,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017955,"completed_at":"2026-09-05T11:59:03.886Z","load_balancing_strategy":"primary","db_duration_s":0.001432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.238821] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.887Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"750f8d1c8da0404922af39cd","created_at":"2026-09-05T11:59:03.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-545e6ff6f9174291","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.283Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-750f8d1c8da0404922af39cd: start","job_status":"start","queue_duration_s":0.603999,"scheduling_latency_s":0.604015} gitlab # [ 317.250941] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.897Z","message":"No existing merge request to be cleaned up."} gitlab # [ 317.252643] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.904Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"750f8d1c8da0404922af39cd","created_at":"2026-09-05T11:59:03.270Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-545e6ff6f9174291","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1b15a9080d4d153034ce9e5bc7c7dfbf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.283Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-750f8d1c8da0404922af39cd: done: 0.017347 sec","job_status":"done","queue_duration_s":0.603999,"scheduling_latency_s":0.604015,"redis_calls":10,"redis_duration_s":0.001237,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000704,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017347,"completed_at":"2026-09-05T11:59:03.904Z","load_balancing_strategy":"primary","db_duration_s":0.001198,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.291315] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21b2a8a61004df62becb4a5b","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21b2a8a61004df62becb4a5b: start","job_status":"start","queue_duration_s":0.36615,"scheduling_latency_s":0.366166} gitlab # [ 317.303739] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21b2a8a61004df62becb4a5b","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21b2a8a61004df62becb4a5b: done: 0.014714 sec","job_status":"done","queue_duration_s":0.36615,"scheduling_latency_s":0.366166,"redis_calls":14,"redis_duration_s":0.001531,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000112,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014714,"completed_at":"2026-09-05T11:59:03.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.344470] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60e389131a6635943d010509","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60e389131a6635943d010509: start","job_status":"start","queue_duration_s":0.381956,"scheduling_latency_s":0.381972} gitlab # [ 317.356685] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60e389131a6635943d010509","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60e389131a6635943d010509: done: 0.013762 sec","job_status":"done","queue_duration_s":0.381956,"scheduling_latency_s":0.381972,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013762,"completed_at":"2026-09-05T11:59:03.935Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.394713] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bf15c39a9bdaf37210b6cb7","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bf15c39a9bdaf37210b6cb7: start","job_status":"start","queue_duration_s":0.396792,"scheduling_latency_s":0.396809} gitlab # [ 317.406892] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bf15c39a9bdaf37210b6cb7","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bf15c39a9bdaf37210b6cb7: done: 0.013934 sec","job_status":"done","queue_duration_s":0.396792,"scheduling_latency_s":0.396809,"redis_calls":13,"redis_duration_s":0.001412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013934,"completed_at":"2026-09-05T11:59:03.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.446023] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c76c4e3740f956c0745bdf73","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c76c4e3740f956c0745bdf73: start","job_status":"start","queue_duration_s":0.412035,"scheduling_latency_s":0.412052} gitlab # [ 317.458323] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c76c4e3740f956c0745bdf73","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c76c4e3740f956c0745bdf73: done: 0.013936 sec","job_status":"done","queue_duration_s":0.412035,"scheduling_latency_s":0.412052,"redis_calls":13,"redis_duration_s":0.001367,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013936,"completed_at":"2026-09-05T11:59:03.965Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.496405] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e942fc9c1de978ac8dc0091","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e942fc9c1de978ac8dc0091: start","job_status":"start","queue_duration_s":0.427009,"scheduling_latency_s":0.427025} gitlab # [ 317.508456] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e942fc9c1de978ac8dc0091","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e942fc9c1de978ac8dc0091: done: 0.015273 sec","job_status":"done","queue_duration_s":0.427009,"scheduling_latency_s":0.427025,"redis_calls":13,"redis_duration_s":0.001417,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015273,"completed_at":"2026-09-05T11:59:03.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.546329] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"597a32cf7c003513230eb3be","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-597a32cf7c003513230eb3be: start","job_status":"start","queue_duration_s":0.443349,"scheduling_latency_s":0.443366} gitlab # [ 317.558565] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"597a32cf7c003513230eb3be","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-597a32cf7c003513230eb3be: done: 0.013739 sec","job_status":"done","queue_duration_s":0.443349,"scheduling_latency_s":0.443366,"redis_calls":13,"redis_duration_s":0.00139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000878,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013739,"completed_at":"2026-09-05T11:59:03.996Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.596745] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:03.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96be2a9e4850531a33431d6d","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96be2a9e4850531a33431d6d: start","job_status":"start","queue_duration_s":0.458137,"scheduling_latency_s":0.458154} gitlab # [ 317.608656] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96be2a9e4850531a33431d6d","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96be2a9e4850531a33431d6d: done: 0.013907 sec","job_status":"done","queue_duration_s":0.458137,"scheduling_latency_s":0.458154,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013907,"completed_at":"2026-09-05T11:59:04.011Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.647831] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ab4643687352b51b2ee87ca","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ab4643687352b51b2ee87ca: start","job_status":"start","queue_duration_s":0.47311,"scheduling_latency_s":0.473126} gitlab # [ 317.660004] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ab4643687352b51b2ee87ca","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ab4643687352b51b2ee87ca: done: 0.013607 sec","job_status":"done","queue_duration_s":0.47311,"scheduling_latency_s":0.473126,"redis_calls":13,"redis_duration_s":0.00136,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013607,"completed_at":"2026-09-05T11:59:04.026Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.698088] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f0633ff438326c2b32ab425","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f0633ff438326c2b32ab425: start","job_status":"start","queue_duration_s":0.487804,"scheduling_latency_s":0.48782} gitlab # [ 317.710125] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f0633ff438326c2b32ab425","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f0633ff438326c2b32ab425: done: 0.013614 sec","job_status":"done","queue_duration_s":0.487804,"scheduling_latency_s":0.48782,"redis_calls":13,"redis_duration_s":0.001362,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013614,"completed_at":"2026-09-05T11:59:04.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.748352] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc44fd9a764deeaa7f1df1fb","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc44fd9a764deeaa7f1df1fb: start","job_status":"start","queue_duration_s":0.502436,"scheduling_latency_s":0.502452} gitlab # [ 317.760277] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc44fd9a764deeaa7f1df1fb","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc44fd9a764deeaa7f1df1fb: done: 0.014307 sec","job_status":"done","queue_duration_s":0.502436,"scheduling_latency_s":0.502452,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014307,"completed_at":"2026-09-05T11:59:04.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.798182] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e41e0d6fd5639ad17558b523","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e41e0d6fd5639ad17558b523: start","job_status":"start","queue_duration_s":0.517788,"scheduling_latency_s":0.517804} gitlab # [ 317.810462] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e41e0d6fd5639ad17558b523","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e41e0d6fd5639ad17558b523: done: 0.014089 sec","job_status":"done","queue_duration_s":0.517788,"scheduling_latency_s":0.517804,"redis_calls":13,"redis_duration_s":0.0013479999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014089,"completed_at":"2026-09-05T11:59:04.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.851996] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68f14b9abfba71e1493aa72a","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68f14b9abfba71e1493aa72a: start","job_status":"start","queue_duration_s":0.532957,"scheduling_latency_s":0.532974} gitlab # [ 317.864507] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68f14b9abfba71e1493aa72a","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68f14b9abfba71e1493aa72a: done: 0.01382 sec","job_status":"done","queue_duration_s":0.532957,"scheduling_latency_s":0.532974,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01382,"completed_at":"2026-09-05T11:59:04.086Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.903986] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7489c5446d1bb8fb307a58e","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7489c5446d1bb8fb307a58e: start","job_status":"start","queue_duration_s":0.547847,"scheduling_latency_s":0.547897} gitlab # [ 317.915932] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7489c5446d1bb8fb307a58e","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7489c5446d1bb8fb307a58e: done: 0.013629 sec","job_status":"done","queue_duration_s":0.547847,"scheduling_latency_s":0.547897,"redis_calls":13,"redis_duration_s":0.0013939999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013629,"completed_at":"2026-09-05T11:59:04.101Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 317.955310] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b059e668108c4dfefb5276c2","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b059e668108c4dfefb5276c2: start","job_status":"start","queue_duration_s":0.562536,"scheduling_latency_s":0.562553} gitlab # [ 317.967895] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b059e668108c4dfefb5276c2","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b059e668108c4dfefb5276c2: done: 0.013808 sec","job_status":"done","queue_duration_s":0.562536,"scheduling_latency_s":0.562553,"redis_calls":13,"redis_duration_s":0.001407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013808,"completed_at":"2026-09-05T11:59:04.116Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.007160] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a929293400f412bfcd302e5c","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a929293400f412bfcd302e5c: start","job_status":"start","queue_duration_s":0.577382,"scheduling_latency_s":0.577398} gitlab # [ 318.019758] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a929293400f412bfcd302e5c","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a929293400f412bfcd302e5c: done: 0.013556 sec","job_status":"done","queue_duration_s":0.577382,"scheduling_latency_s":0.577398,"redis_calls":13,"redis_duration_s":0.001363,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013556,"completed_at":"2026-09-05T11:59:04.130Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.059900] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e3ddeee59b8579248f66030","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e3ddeee59b8579248f66030: start","job_status":"start","queue_duration_s":0.592001,"scheduling_latency_s":0.592017} gitlab # [ 318.072690] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e3ddeee59b8579248f66030","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e3ddeee59b8579248f66030: done: 0.013547 sec","job_status":"done","queue_duration_s":0.592001,"scheduling_latency_s":0.592017,"redis_calls":13,"redis_duration_s":0.001338,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013547,"completed_at":"2026-09-05T11:59:04.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.110648] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e81806df627e6f2a64af0fe2","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e81806df627e6f2a64af0fe2: start","job_status":"start","queue_duration_s":0.60659,"scheduling_latency_s":0.606605} gitlab # [ 318.122618] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e81806df627e6f2a64af0fe2","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e81806df627e6f2a64af0fe2: done: 0.014035 sec","job_status":"done","queue_duration_s":0.60659,"scheduling_latency_s":0.606605,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014035,"completed_at":"2026-09-05T11:59:04.160Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.160265] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"490ca06efa860fda05c4543f","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-490ca06efa860fda05c4543f: start","job_status":"start","queue_duration_s":0.621662,"scheduling_latency_s":0.621678} gitlab # [ 318.172224] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"490ca06efa860fda05c4543f","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-490ca06efa860fda05c4543f: done: 0.014089 sec","job_status":"done","queue_duration_s":0.621662,"scheduling_latency_s":0.621678,"redis_calls":13,"redis_duration_s":0.001416,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014089,"completed_at":"2026-09-05T11:59:04.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.210246] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb3d00cc0130d69902389246","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb3d00cc0130d69902389246: start","job_status":"start","queue_duration_s":0.636794,"scheduling_latency_s":0.636811} gitlab # [ 318.222321] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb3d00cc0130d69902389246","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb3d00cc0130d69902389246: done: 0.013513 sec","job_status":"done","queue_duration_s":0.636794,"scheduling_latency_s":0.636811,"redis_calls":13,"redis_duration_s":0.001346,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013513,"completed_at":"2026-09-05T11:59:04.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.261042] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9d9f8c6057ded0dc4d3e632","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9d9f8c6057ded0dc4d3e632: start","job_status":"start","queue_duration_s":0.651364,"scheduling_latency_s":0.65138} gitlab # [ 318.273298] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9d9f8c6057ded0dc4d3e632","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9d9f8c6057ded0dc4d3e632: done: 0.013545 sec","job_status":"done","queue_duration_s":0.651364,"scheduling_latency_s":0.65138,"redis_calls":13,"redis_duration_s":0.001353,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013545,"completed_at":"2026-09-05T11:59:04.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.312057] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"016d06ad1d087c3641fa87ae","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-016d06ad1d087c3641fa87ae: start","job_status":"start","queue_duration_s":0.665948,"scheduling_latency_s":0.665963} gitlab # [ 318.324718] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"016d06ad1d087c3641fa87ae","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-016d06ad1d087c3641fa87ae: done: 0.01368 sec","job_status":"done","queue_duration_s":0.665948,"scheduling_latency_s":0.665963,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01368,"completed_at":"2026-09-05T11:59:04.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.363923] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cac3c36a2143a8c25867d942","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cac3c36a2143a8c25867d942: start","job_status":"start","queue_duration_s":0.680709,"scheduling_latency_s":0.680725} gitlab # [ 318.376536] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cac3c36a2143a8c25867d942","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cac3c36a2143a8c25867d942: done: 0.013546 sec","job_status":"done","queue_duration_s":0.680709,"scheduling_latency_s":0.680725,"redis_calls":13,"redis_duration_s":0.001327,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013546,"completed_at":"2026-09-05T11:59:04.233Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.415129] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b889034e4ad3abd9d8dce9e9","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b889034e4ad3abd9d8dce9e9: start","job_status":"start","queue_duration_s":0.695384,"scheduling_latency_s":0.695401} gitlab # [ 318.427648] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b889034e4ad3abd9d8dce9e9","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b889034e4ad3abd9d8dce9e9: done: 0.013677 sec","job_status":"done","queue_duration_s":0.695384,"scheduling_latency_s":0.695401,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000876,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013677,"completed_at":"2026-09-05T11:59:04.248Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.467368] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af44c7ac498487f162ef344e","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af44c7ac498487f162ef344e: start","job_status":"start","queue_duration_s":0.710124,"scheduling_latency_s":0.71014} gitlab # [ 318.479875] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af44c7ac498487f162ef344e","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af44c7ac498487f162ef344e: done: 0.013851 sec","job_status":"done","queue_duration_s":0.710124,"scheduling_latency_s":0.71014,"redis_calls":13,"redis_duration_s":0.001365,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013851,"completed_at":"2026-09-05T11:59:04.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.518549] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6996f984e65f873ddc5e3e5","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6996f984e65f873ddc5e3e5: start","job_status":"start","queue_duration_s":0.725023,"scheduling_latency_s":0.72504} gitlab # [ 318.530907] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6996f984e65f873ddc5e3e5","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6996f984e65f873ddc5e3e5: done: 0.014164 sec","job_status":"done","queue_duration_s":0.725023,"scheduling_latency_s":0.72504,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000866,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014164,"completed_at":"2026-09-05T11:59:04.278Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.569213] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"051a2bbf6226a69658ff43c8","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-051a2bbf6226a69658ff43c8: start","job_status":"start","queue_duration_s":0.740286,"scheduling_latency_s":0.740303} gitlab # [ 318.581190] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"051a2bbf6226a69658ff43c8","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-051a2bbf6226a69658ff43c8: done: 0.050051 sec","job_status":"done","queue_duration_s":0.740286,"scheduling_latency_s":0.740303,"redis_calls":13,"redis_duration_s":0.0015530000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.050051,"completed_at":"2026-09-05T11:59:04.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.620138] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3acba141d6c90c563d115ab5","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3acba141d6c90c563d115ab5: start","job_status":"start","queue_duration_s":0.79145,"scheduling_latency_s":0.791467} gitlab # [ 318.631840] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3acba141d6c90c563d115ab5","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3acba141d6c90c563d115ab5: done: 0.013631 sec","job_status":"done","queue_duration_s":0.79145,"scheduling_latency_s":0.791467,"redis_calls":13,"redis_duration_s":0.0014039999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013631,"completed_at":"2026-09-05T11:59:04.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.670053] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"118c0a1aee39ec93d7edc7cd","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-118c0a1aee39ec93d7edc7cd: start","job_status":"start","queue_duration_s":0.807051,"scheduling_latency_s":0.807068} gitlab # [ 318.683171] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"118c0a1aee39ec93d7edc7cd","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-118c0a1aee39ec93d7edc7cd: done: 0.014715 sec","job_status":"done","queue_duration_s":0.807051,"scheduling_latency_s":0.807068,"redis_calls":13,"redis_duration_s":0.0023610000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014715,"completed_at":"2026-09-05T11:59:04.361Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.722029] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88dfaa2cc764c77b3b0f9085","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88dfaa2cc764c77b3b0f9085: start","job_status":"start","queue_duration_s":0.822766,"scheduling_latency_s":0.822782} gitlab # [ 318.734434] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88dfaa2cc764c77b3b0f9085","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88dfaa2cc764c77b3b0f9085: done: 0.016009 sec","job_status":"done","queue_duration_s":0.822766,"scheduling_latency_s":0.822782,"redis_calls":13,"redis_duration_s":0.001554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016009,"completed_at":"2026-09-05T11:59:04.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.773453] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6607ba5b9df343156434c9e9","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6607ba5b9df343156434c9e9: start","job_status":"start","queue_duration_s":0.839831,"scheduling_latency_s":0.839846} gitlab # [ 318.785602] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6607ba5b9df343156434c9e9","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6607ba5b9df343156434c9e9: done: 0.01439 sec","job_status":"done","queue_duration_s":0.839831,"scheduling_latency_s":0.839846,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01439,"completed_at":"2026-09-05T11:59:04.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.824125] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4ee3799df0cb3b4f630832a","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4ee3799df0cb3b4f630832a: start","job_status":"start","queue_duration_s":0.85624,"scheduling_latency_s":0.856257} gitlab # [ 318.836622] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4ee3799df0cb3b4f630832a","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4ee3799df0cb3b4f630832a: done: 0.014473 sec","job_status":"done","queue_duration_s":0.85624,"scheduling_latency_s":0.856257,"redis_calls":13,"redis_duration_s":0.00141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014473,"completed_at":"2026-09-05T11:59:04.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.875615] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6009d22a2d513f65b63bfce","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6009d22a2d513f65b63bfce: start","job_status":"start","queue_duration_s":0.871752,"scheduling_latency_s":0.871767} gitlab # [ 318.888412] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6009d22a2d513f65b63bfce","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6009d22a2d513f65b63bfce: done: 0.014341 sec","job_status":"done","queue_duration_s":0.871752,"scheduling_latency_s":0.871767,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014341,"completed_at":"2026-09-05T11:59:04.425Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.927975] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc29f4e8122584d5c58e2a82","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc29f4e8122584d5c58e2a82: start","job_status":"start","queue_duration_s":0.887124,"scheduling_latency_s":0.88714} gitlab # [ 318.940627] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc29f4e8122584d5c58e2a82","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc29f4e8122584d5c58e2a82: done: 0.014267 sec","job_status":"done","queue_duration_s":0.887124,"scheduling_latency_s":0.88714,"redis_calls":13,"redis_duration_s":0.001382,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014267,"completed_at":"2026-09-05T11:59:04.441Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 318.979447] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc08797ef762dadc658b1380","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc08797ef762dadc658b1380: start","job_status":"start","queue_duration_s":0.902422,"scheduling_latency_s":0.902438} gitlab # [ 318.991796] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc08797ef762dadc658b1380","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc08797ef762dadc658b1380: done: 0.014628 sec","job_status":"done","queue_duration_s":0.902422,"scheduling_latency_s":0.902438,"redis_calls":13,"redis_duration_s":0.001347,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014628,"completed_at":"2026-09-05T11:59:04.456Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.031098] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2e6990be816d5dde840b611","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2e6990be816d5dde840b611: start","job_status":"start","queue_duration_s":0.920213,"scheduling_latency_s":0.92023} gitlab # [ 319.043489] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2e6990be816d5dde840b611","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2e6990be816d5dde840b611: done: 0.014862 sec","job_status":"done","queue_duration_s":0.920213,"scheduling_latency_s":0.92023,"redis_calls":13,"redis_duration_s":0.001384,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014862,"completed_at":"2026-09-05T11:59:04.474Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.083813] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aabc70e54e1bde5c1f93d513","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aabc70e54e1bde5c1f93d513: start","job_status":"start","queue_duration_s":0.93612,"scheduling_latency_s":0.936135} gitlab # [ 319.096593] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aabc70e54e1bde5c1f93d513","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aabc70e54e1bde5c1f93d513: done: 0.014407 sec","job_status":"done","queue_duration_s":0.93612,"scheduling_latency_s":0.936135,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014407,"completed_at":"2026-09-05T11:59:04.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.135228] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae895a9a872065c49dfa9b46","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae895a9a872065c49dfa9b46: start","job_status":"start","queue_duration_s":0.951561,"scheduling_latency_s":0.951577} gitlab # [ 319.147682] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae895a9a872065c49dfa9b46","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae895a9a872065c49dfa9b46: done: 0.014376 sec","job_status":"done","queue_duration_s":0.951561,"scheduling_latency_s":0.951577,"redis_calls":13,"redis_duration_s":0.001382,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014376,"completed_at":"2026-09-05T11:59:04.505Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.186345] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da1dda9b954d322384e1b6ac","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da1dda9b954d322384e1b6ac: start","job_status":"start","queue_duration_s":0.967039,"scheduling_latency_s":0.967055} gitlab # [ 319.198479] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da1dda9b954d322384e1b6ac","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da1dda9b954d322384e1b6ac: done: 0.014527 sec","job_status":"done","queue_duration_s":0.967039,"scheduling_latency_s":0.967055,"redis_calls":13,"redis_duration_s":0.0014060000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014527,"completed_at":"2026-09-05T11:59:04.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.236762] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ae4ac44e378d9458eb4ff89","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ae4ac44e378d9458eb4ff89: start","job_status":"start","queue_duration_s":0.98338,"scheduling_latency_s":0.983397} gitlab # [ 319.248727] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ae4ac44e378d9458eb4ff89","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ae4ac44e378d9458eb4ff89: done: 0.015215 sec","job_status":"done","queue_duration_s":0.98338,"scheduling_latency_s":0.983397,"redis_calls":13,"redis_duration_s":0.001389,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015215,"completed_at":"2026-09-05T11:59:04.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.287641] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54f27b82d1400c88a6fcac26","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54f27b82d1400c88a6fcac26: start","job_status":"start","queue_duration_s":0.999623,"scheduling_latency_s":0.999638} gitlab # [ 319.299797] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54f27b82d1400c88a6fcac26","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54f27b82d1400c88a6fcac26: done: 0.014842 sec","job_status":"done","queue_duration_s":0.999623,"scheduling_latency_s":0.999638,"redis_calls":13,"redis_duration_s":0.001392,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014842,"completed_at":"2026-09-05T11:59:04.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.338029] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25106bcfae0bf0ca13ef300b","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25106bcfae0bf0ca13ef300b: start","job_status":"start","queue_duration_s":1.01551,"scheduling_latency_s":1.015526} gitlab # [ 319.350166] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25106bcfae0bf0ca13ef300b","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25106bcfae0bf0ca13ef300b: done: 0.015181 sec","job_status":"done","queue_duration_s":1.01551,"scheduling_latency_s":1.015526,"redis_calls":13,"redis_duration_s":0.001411,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015181,"completed_at":"2026-09-05T11:59:04.570Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.388101] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96a0c3429b42c94fd577f8ab","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96a0c3429b42c94fd577f8ab: start","job_status":"start","queue_duration_s":1.031916,"scheduling_latency_s":1.031934} gitlab # [ 319.400079] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96a0c3429b42c94fd577f8ab","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96a0c3429b42c94fd577f8ab: done: 0.015304 sec","job_status":"done","queue_duration_s":1.031916,"scheduling_latency_s":1.031934,"redis_calls":13,"redis_duration_s":0.001361,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015304,"completed_at":"2026-09-05T11:59:04.586Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.437848] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4951234d4ea409fd2aadd7f1","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4951234d4ea409fd2aadd7f1: start","job_status":"start","queue_duration_s":1.048272,"scheduling_latency_s":1.048287} gitlab # [ 319.449966] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4951234d4ea409fd2aadd7f1","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4951234d4ea409fd2aadd7f1: done: 0.0146 sec","job_status":"done","queue_duration_s":1.048272,"scheduling_latency_s":1.048287,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0146,"completed_at":"2026-09-05T11:59:04.602Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.489208] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8759166d155adf29564e194b","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8759166d155adf29564e194b: start","job_status":"start","queue_duration_s":1.063942,"scheduling_latency_s":1.063958} gitlab # [ 319.501044] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8759166d155adf29564e194b","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8759166d155adf29564e194b: done: 0.014771 sec","job_status":"done","queue_duration_s":1.063942,"scheduling_latency_s":1.063958,"redis_calls":13,"redis_duration_s":0.001394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014771,"completed_at":"2026-09-05T11:59:04.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.539092] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c309a67412aa45915cfca3dc","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c309a67412aa45915cfca3dc: start","job_status":"start","queue_duration_s":1.079766,"scheduling_latency_s":1.079782} gitlab # [ 319.551346] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c309a67412aa45915cfca3dc","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c309a67412aa45915cfca3dc: done: 0.014484 sec","job_status":"done","queue_duration_s":1.079766,"scheduling_latency_s":1.079782,"redis_calls":13,"redis_duration_s":0.001371,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014484,"completed_at":"2026-09-05T11:59:04.633Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.589435] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe1c3e8c7e5e2e5cbcc45896","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe1c3e8c7e5e2e5cbcc45896: start","job_status":"start","queue_duration_s":1.096152,"scheduling_latency_s":1.096169} gitlab # [ 319.601593] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe1c3e8c7e5e2e5cbcc45896","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe1c3e8c7e5e2e5cbcc45896: done: 0.014651 sec","job_status":"done","queue_duration_s":1.096152,"scheduling_latency_s":1.096169,"redis_calls":13,"redis_duration_s":0.0013930000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014651,"completed_at":"2026-09-05T11:59:04.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.639652] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f63e157dbb74093f8873fa58","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f63e157dbb74093f8873fa58: start","job_status":"start","queue_duration_s":1.11186,"scheduling_latency_s":1.111896} gitlab # [ 319.651812] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f63e157dbb74093f8873fa58","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f63e157dbb74093f8873fa58: done: 0.015879 sec","job_status":"done","queue_duration_s":1.11186,"scheduling_latency_s":1.111896,"redis_calls":13,"redis_duration_s":0.002579,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015879,"completed_at":"2026-09-05T11:59:04.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.690932] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c43c584318aca79a00961ee6","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c43c584318aca79a00961ee6: start","job_status":"start","queue_duration_s":1.128805,"scheduling_latency_s":1.12882} gitlab # [ 319.703088] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c43c584318aca79a00961ee6","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c43c584318aca79a00961ee6: done: 0.015982 sec","job_status":"done","queue_duration_s":1.128805,"scheduling_latency_s":1.12882,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015982,"completed_at":"2026-09-05T11:59:04.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.741137] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"001b4ae93c8c98cd1ceddafa","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-001b4ae93c8c98cd1ceddafa: start","job_status":"start","queue_duration_s":1.145893,"scheduling_latency_s":1.145909} gitlab # [ 319.753212] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"001b4ae93c8c98cd1ceddafa","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-001b4ae93c8c98cd1ceddafa: done: 0.014386 sec","job_status":"done","queue_duration_s":1.145893,"scheduling_latency_s":1.145909,"redis_calls":13,"redis_duration_s":0.001399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014386,"completed_at":"2026-09-05T11:59:04.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 319.791314] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8994d62d2963c333195c7d30","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8994d62d2963c333195c7d30: start","job_status":"start","queue_duration_s":1.16219,"scheduling_latency_s":1.162206} gitlab # [ 319.803288] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:04.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8994d62d2963c333195c7d30","created_at":"2026-09-05T11:59:03.420Z","trace_propagation_headers":{"sentry-trace":"7a9cdd1ffa6148d290b22ebafdba478b-0f4219901571432a","baggage":"sentry-trace_id=7a9cdd1ffa6148d290b22ebafdba478b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b17b961ea6e93979fda68b633a656c59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:03.539Z","job_size_bytes":2,"pid":2037,"sidekiq_tid":"maot","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8994d62d2963c333195c7d30: done: 0.014497 sec","job_status":"done","queue_duration_s":1.16219,"scheduling_latency_s":1.162206,"redis_calls":13,"redis_duration_s":0.001371,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014497,"completed_at":"2026-09-05T11:59:04.716Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 322.614569] runuser[2980]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 322.616341] setup-gitlab-start[2949]: + systemctl restart gitlab-sidekiq.service gitlab # [ 322.656688] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 322.658268] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:09.539Z","message":"Shutting down"} gitlab # [ 322.660106] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:09.541Z","message":"Scheduler exiting..."} gitlab # [ 322.661954] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:09.541Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 322.664177] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:09.542Z","message":"Scheduler exiting..."} gitlab # [ 323.164847] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:10.048Z","message":"Bye!"} gitlab # [ 323.976203] sidekiq-cluster[2037]: {"severity":"INFO","time":"2026-09-05T11:59:10.858Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2037,"worker_id":"sidekiq_0","memwd_rss_bytes":956022784,"retry":0} gitlab # [ 325.239421] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 325.239811] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 325.239982] systemd[1]: gitlab-sidekiq.service: Consumed 1min 53.229s CPU time over 3min 21.425s wall clock time, 947.9M memory peak, 4.4M read from disk. gitlab # [ 325.253273] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 325.259277] setup-gitlab-start[2949]: + mkdir -p /var/lib/secrets gitlab # [ 325.264280] setup-gitlab-start[2949]: + echo nixbot-test-token-of-20-chars gitlab # [ 325.267129] setup-gitlab-start[2949]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 325.270919] setup-gitlab-start[2949]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 327.023595] gitaly[897]: time="2026-09-05T11:59:13.905Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:23.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:13.905" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.032497] gitaly[897]: time="2026-09-05T11:59:13.906Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:23.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:13.905" grpc.time_ms=0.943 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.335881] gitaly[897]: time="2026-09-05T11:59:14.219Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.218" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.394433] gitaly[897]: time="2026-09-05T11:59:14.219Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.218" grpc.time_ms=0.638 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.482249] gitaly[897]: time="2026-09-05T11:59:14.365Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 diskcache=b29bf9f9-e4c5-409d-bebf-9f26765f8332 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-05T11:59:24.084" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.295" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.513485] gitaly[897]: time="2026-09-05T11:59:14.366Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=37 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=43 command.spawn_token_fork_ms=6 command.system_time_ms=31 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-05T11:59:24.084" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.295" grpc.time_ms=70.246 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.545167] gitaly[897]: time="2026-09-05T11:59:14.391Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.179" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.389" pid=897 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 327.563351] gitaly[897]: time="2026-09-05T11:59:14.391Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.179" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.389" grpc.time_ms=1.559 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.580413] gitaly[897]: time="2026-09-05T11:59:14.420Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-05T11:59:24.191" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.401" grpc.time_ms=18.545 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.663575] gitaly[897]: time="2026-09-05T11:59:14.546Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.546" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.694693] gitaly[897]: time="2026-09-05T11:59:14.547Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.546" grpc.time_ms=0.663 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.727647] gitaly[897]: time="2026-09-05T11:59:14.599Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 diskcache=c7159077-88ad-464d-a226-ededaa5a0942 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-05T11:59:24.009" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.551" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.749610] gitaly[897]: time="2026-09-05T11:59:14.600Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1440 command.oublock=32 command.real_time_ms=29 command.spawn_token_fork_ms=5 command.system_time_ms=19 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-05T11:59:24.009" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.551" grpc.time_ms=49.038 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.771131] gitaly[897]: time="2026-09-05T11:59:14.609Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.066" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.607" pid=897 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 327.785975] gitaly[897]: time="2026-09-05T11:59:14.609Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.066" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.607" grpc.time_ms=1.31 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.801345] gitaly[897]: time="2026-09-05T11:59:14.623Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-05T11:59:24.073" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.613" grpc.time_ms=9.477 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.819205] gitaly[897]: time="2026-09-05T11:59:14.665Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T11:59:24.123" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.664" grpc.time_ms=0.285 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.832713] gitaly[897]: time="2026-09-05T11:59:14.674Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-05T11:59:24.132" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:14.674" grpc.time_ms=0.531 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.397775] gitaly[897]: time="2026-09-05T11:59:15.280Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZQSQPMMRAX3ZN4HV4MY9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-05T11:59:25.052" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:15.263" grpc.time_ms=16.669 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.918809] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:15 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 3617 gitlab # [ 328.925987] setup-gitlab-start[2949]: + mkdir -p /tmp/test-flake gitlab # [ 328.928698] setup-gitlab-start[2949]: + cd /tmp/test-flake gitlab # [ 328.931036] setup-gitlab-start[2949]: + git init -b master gitlab # [ 328.946420] setup-gitlab-start[3430]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 328.950006] setup-gitlab-start[2949]: + git config user.name test gitlab # [ 328.952700] setup-gitlab-start[2949]: + git config user.email test@example.com gitlab # [ 328.955672] setup-gitlab-start[2949]: + cat gitlab # [ 328.958501] setup-gitlab-start[2949]: + git add flake.nix gitlab # [ 328.963263] setup-gitlab-start[2949]: + git commit -m 'initial commit' gitlab # [ 328.985300] setup-gitlab-start[3435]: [master (root-commit) 8a22cc6] initial commit gitlab # [ 328.989433] setup-gitlab-start[3435]: 1 file changed, 10 insertions(+) gitlab # [ 328.992313] setup-gitlab-start[3435]: create mode 100644 flake.nix gitlab # [ 328.994980] setup-gitlab-start[2949]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 329.000447] setup-gitlab-start[2949]: + git push -u origin master gitlab # [ 329.356973] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:16 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 313 gitlab # [ 329.588204] gitaly[897]: time="2026-09-05T11:59:16.470Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZVRH45NP0DK60KHPF7RV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:16.452" grpc.time_ms=18.141 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 329.627375] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:16 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 230 gitlab # [ 330.237095] gitaly[897]: time="2026-09-05T11:59:17.120Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-05T11:59:27.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:17.103" grpc.time_ms=16.655 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 330.325261] gitaly[897]: time="2026-09-05T11:59:17.208Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-05T11:59:47.088" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-05T11:59:17.188" grpc.time_ms=19.427 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 330.376396] gitaly[897]: time="2026-09-05T11:59:17.259Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1RPZW09FP863AAXMJ1HX43F duration_ms=302ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 330.386568] sidekiq-cluster[3377]: {"severity":"INFO","time":"2026-09-05T11:59:17.240Z","message":"Starting cluster with 1 processes"} gitlab # [ 330.558515] gitaly[897]: time="2026-09-05T11:59:17.441Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1RPZW09FP863AAXMJ1HX43F duration_ms=180ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 330.567686] gitaly[897]: time="2026-09-05T11:59:17.442Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:16.954" grpc.time_ms=487.694 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 330.615865] gitaly[897]: time="2026-09-05T11:59:17.498Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:17.497" grpc.time_ms=0.916 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 330.673270] gitaly[897]: time="2026-09-05T11:59:17.556Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:17.554" grpc.time_ms=1.614 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 330.756081] gitaly[897]: time="2026-09-05T11:59:17.639Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:17.638" grpc.time_ms=0.834 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 331.043605] gitaly[897]: time="2026-09-05T11:59:17.926Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-05T11:59:27.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:17.911" grpc.time_ms=15.247 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 331.136359] gitaly[897]: time="2026-09-05T11:59:18.019Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=30 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=5 command.system_time_ms=27 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-05T11:59:27.077" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:17.978" grpc.time_ms=41.082 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 331.182296] gitaly[897]: time="2026-09-05T11:59:18.065Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1RPZW09FP863AAXMJ1HX43F duration_ms=344ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 331.194248] gitaly[897]: time="2026-09-05T11:59:18.067Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:17.720" grpc.time_ms=346.7 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 331.220585] gitaly[897]: time="2026-09-05T11:59:18.084Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F diskcache=2b98cbca-aee2-4fd3-a84b-5c9ec1d7f442 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:16.845" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 331.240902] systemd[1]: Finished setup-gitlab.service. gitlab # [ 331.242647] gitlab-workhorse[823]: time="2026-09-05T11:59:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 331.245754] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:18 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1596 gitlab # [ 331.249010] setup-gitlab-start[3439]: To http://gitlab/root/test-flake.git gitlab # [ 331.250421] setup-gitlab-start[3439]: * [new branch] master -> master gitlab # [ 331.251798] setup-gitlab-start[3439]: branch 'master' set up to track 'origin/master'. gitlab # [ 331.253378] gitaly[897]: time="2026-09-05T11:59:18.085Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=357 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13718 command.oublock=32 command.real_time_ms=1231 command.spawn_token_fork_ms=1 command.system_time_ms=191 command.user_time_ms=166 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:16.845" grpc.time_ms=1239.542 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 331.271253] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 332.14 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 332.403357] nixbot[3535]: {"ts": "2026-09-05T11:59:19.285Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 332.427696] nixbot[3535]: {"ts": "2026-09-05T11:59:19.310Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 332.444485] nixbot[3535]: {"ts": "2026-09-05T11:59:19.327Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 332.451120] nixbot[3535]: {"ts": "2026-09-05T11:59:19.332Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 332.457822] nixbot[3535]: {"ts": "2026-09-05T11:59:19.337Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 332.464196] nixbot[3535]: {"ts": "2026-09-05T11:59:19.342Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 332.470579] nixbot[3535]: {"ts": "2026-09-05T11:59:19.347Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 332.477180] nixbot[3535]: {"ts": "2026-09-05T11:59:19.349Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 332.483484] nixbot[3535]: {"ts": "2026-09-05T11:59:19.354Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 332.490684] nixbot[3535]: {"ts": "2026-09-05T11:59:19.356Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 332.497656] nixbot[3535]: {"ts": "2026-09-05T11:59:19.361Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 332.504177] nixbot[3535]: {"ts": "2026-09-05T11:59:19.367Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 332.510711] nixbot[3535]: {"ts": "2026-09-05T11:59:19.370Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 332.517195] nixbot[3535]: {"ts": "2026-09-05T11:59:19.372Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 332.523464] nixbot[3535]: {"ts": "2026-09-05T11:59:19.376Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 332.529744] nixbot[3535]: {"ts": "2026-09-05T11:59:19.379Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 332.535758] nixbot[3535]: {"ts": "2026-09-05T11:59:19.381Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 332.541169] nixbot[3535]: {"ts": "2026-09-05T11:59:19.382Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 332.545967] nixbot[3535]: {"ts": "2026-09-05T11:59:19.387Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 332.550536] nixbot[3535]: {"ts": "2026-09-05T11:59:19.389Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 332.555044] nixbot[3535]: {"ts": "2026-09-05T11:59:19.391Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 332.559348] nixbot[3535]: {"ts": "2026-09-05T11:59:19.400Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 332.563955] nixbot[3535]: {"ts": "2026-09-05T11:59:19.402Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 332.568565] nixbot[3535]: {"ts": "2026-09-05T11:59:19.405Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 332.572624] nixbot[3535]: {"ts": "2026-09-05T11:59:19.406Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 332.576960] nixbot[3535]: {"ts": "2026-09-05T11:59:19.410Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 332.581165] nixbot[3535]: {"ts": "2026-09-05T11:59:19.413Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 332.585061] nixbot[3535]: {"ts": "2026-09-05T11:59:19.414Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 332.588592] nixbot[3535]: {"ts": "2026-09-05T11:59:19.416Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 332.592347] nixbot[3535]: {"ts": "2026-09-05T11:59:19.439Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 332.595558] nixbot[3535]: {"ts": "2026-09-05T11:59:19.447Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 332.598931] nixbot[3535]: {"ts": "2026-09-05T11:59:19.449Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 332.602962] nixbot[3535]: {"ts": "2026-09-05T11:59:19.452Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 332.947904] nixbot[3535]: INFO: Started server process [3535] gitlab # [ 332.950642] nixbot[3535]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 332.975384] nixbot[3535]: INFO: ::ffff:127.0.0.1:59072 - "GET /health HTTP/1.1" 200 OK gitlab # [ 332.987863] systemd[1]: Started nixbot CI service. gitlab # [ 332.992568] systemd[1]: Reached target Multi-User System. gitlab # [ 332.994927] systemd[1]: Startup finished in 791ms (kernel) + 3.375s (initrd) + 5min 28.826s (userspace) = 5min 32.992s. gitlab # [ 333.272549] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:20 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 293 gitlab # [ 333.274825] nixbot[3535]: {"ts": "2026-09-05T11:59:20.154Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 333.315681] nixbot[3535]: {"ts": "2026-09-05T11:59:20.198Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 333.631393] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 302 gitlab # [ 333.634026] nixbot[3535]: {"ts": "2026-09-05T11:59:20.516Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 333.636902] nixbot[3535]: {"ts": "2026-09-05T11:59:20.520Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab: (finished: waiting for unit nixbot.service, in 2.14 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 333.849225] nixbot[3535]: INFO: ::ffff:127.0.0.1:59086 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 333.941568] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 67 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) gitlab # [ 334.011767] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 370 gitlab # [ 334.018579] nixbot[3535]: {"ts": "2026-09-05T11:59:20.896Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 335.012209] nixbot[3535]: {"ts": "2026-09-05T11:59:21.894Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 335.021190] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:21 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 991 gitlab # [ 335.093663] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 71 gitlab # [ 335.109409] nixbot[3535]: {"ts": "2026-09-05T11:59:21.977Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 335.240098] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 238 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.30 seconds) (finished: subtest: project discovered and webhook registered, in 1.40 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 335.386614] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 279 gitlab # [ 335.390430] nixbot[3535]: {"ts": "2026-09-05T11:59:22.272Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 335.394367] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 56 gitlab # [ 335.503118] gitaly[897]: time="2026-09-05T11:59:22.385Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01N7RZEFDKAY2DSJ5MW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.374" grpc.time_ms=11.652 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # Enumerating objects: 5, done. gitlab # [ 335.524410] gitaly[897]: time="2026-09-05T11:59:22.405Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=8 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ01NFJSQS3VC6RWDHS0H3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-05T11:59:52.271" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:22.372" grpc.time_ms=32.606 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # Counting objects: 20% (1/5)[ 335.543143] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 107 gitlab # Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 335.558184] gitaly[897]: time="2026-09-05T11:59:22.441Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ01NFJSQS3VC6RWDHS0H3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T11:59:52.339" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T11:59:22.440" grpc.time_ms=1.274 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.580081] gitaly[897]: time="2026-09-05T11:59:22.462Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ01NFJSQS3VC6RWDHS0H3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-05T11:59:32.242" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T11:59:22.452" grpc.time_ms=9.271 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.636339] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 232 gitlab # [ 335.640515] nixbot[3535]: {"ts": "2026-09-05T11:59:22.521Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 335.736198] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 91 gitlab # [ 335.747229] nixbot[3535]: {"ts": "2026-09-05T11:59:22.620Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 335.753742] nixbot[3535]: {"ts": "2026-09-05T11:59:22.625Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 335.846648] gitaly[897]: time="2026-09-05T11:59:22.730Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ021JT62NWGSTPPPW7Z91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.719" grpc.time_ms=9.976 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 335.874237] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 56 gitlab # [ 335.890137] gitaly[897]: time="2026-09-05T11:59:22.773Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=25 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-05T11:59:52.633" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-05T11:59:22.733" grpc.time_ms=39.513 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.955675] gitaly[897]: time="2026-09-05T11:59:22.838Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 duration_ms=211ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 336.012235] gitaly[897]: time="2026-09-05T11:59:22.895Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02492439E7MDNDN37DD2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.880" pid=897 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 336.045571] gitaly[897]: time="2026-09-05T11:59:22.895Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02492439E7MDNDN37DD2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.880" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.072642] gitlab-workhorse[823]: time="2026-09-05T11:59:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 336.075657] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 134 gitlab # [ 336.079856] gitaly[897]: time="2026-09-05T11:59:22.896Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02492439E7MDNDN37DD2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.880" grpc.time_ms=14.923 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.104609] gitaly[897]: time="2026-09-05T11:59:22.915Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 duration_ms=75ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 336.109429] gitaly[897]: time="2026-09-05T11:59:22.916Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:22.619" grpc.time_ms=296.574 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 336.126325] gitaly[897]: time="2026-09-05T11:59:22.975Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:22.974" grpc.time_ms=0.751 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 336.142441] gitaly[897]: time="2026-09-05T11:59:22.995Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ028N65Q7T2ZSNG8P45KZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.961" pid=897 remote_ip="2001:db8:1::1" request_sha=54899fd9297d042e0d2a554bbc47a395452bce05 response_bytes=229 span.kind=server system=grpc gitlab # [ 336.156964] gitlab-workhorse[823]: time="2026-09-05T11:59:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=229 gitlab # [ 336.159336] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:22 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 229 "" "git/2.55.0" 95 gitlab # [ 336.161822] gitaly[897]: time="2026-09-05T11:59:22.995Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=13 command.system_time_ms=0 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ028N65Q7T2ZSNG8P45KZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.961" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.177761] gitaly[897]: time="2026-09-05T11:59:22.995Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=13 command.system_time_ms=0 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ028N65Q7T2ZSNG8P45KZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.961" grpc.time_ms=34.255 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.193473] gitaly[897]: time="2026-09-05T11:59:23.024Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:23.023" grpc.time_ms=0.688 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 336.203903] gitaly[897]: time="2026-09-05T11:59:23.063Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:23.063" grpc.time_ms=0.463 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 336.226077] gitaly[897]: time="2026-09-05T11:59:23.109Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02D1EFNM6QDG2MFDZK4Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.101" grpc.time_ms=8.038 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.239413] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:23 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 68 gitlab # [ 336.283878] gitaly[897]: time="2026-09-05T11:59:23.167Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 duration_ms=76ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 336.289437] gitaly[897]: time="2026-09-05T11:59:23.167Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:23.090" grpc.time_ms=77.638 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 336.305805] gitaly[897]: time="2026-09-05T11:59:23.174Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02F7EC1W50RQMF1RWBMP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.162" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # 8a22cc6..3dfbb55 [ 336.319931] gitlab-workhorse[823]: time="2026-09-05T11:59:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # master -> master[ 336.322392] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 63 gitlab # gitlab # [ 336.324803] gitlab-workhorse[823]: time="2026-09-05T11:59:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 336.327401] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:23 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 732 gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.09 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab # [ 336.330197] gitaly[897]: time="2026-09-05T11:59:23.174Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02F7EC1W50RQMF1RWBMP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.162" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.346277] gitaly[897]: time="2026-09-05T11:59:23.174Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02F7EC1W50RQMF1RWBMP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.162" grpc.time_ms=12.352 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 336.363783] gitaly[897]: time="2026-09-05T11:59:23.175Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 diskcache=6f309b96-4c4c-40f7-be07-43d5dcf2354a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.527" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 336.375597] gitaly[897]: time="2026-09-05T11:59:23.175Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=211 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13133 command.oublock=32 command.real_time_ms=638 command.spawn_token_fork_ms=2 command.system_time_ms=108 command.user_time_ms=103 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:22.527" grpc.time_ms=648.481 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 336.391909] gitaly[897]: time="2026-09-05T11:59:23.269Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02H8BWEKY5SSC2PFADBN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.250" pid=897 remote_ip="2001:db8:1::1" request_sha=43e6a369ac049278598e69b2e7a5175899f1ecda response_bytes=339 span.kind=server system=grpc gitlab # [ 336.403027] gitlab-workhorse[823]: time="2026-09-05T11:59:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=339 gitlab # [ 336.404935] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 339 "" "git/2.55.0" 94 gitlab # [ 336.406801] gitaly[897]: time="2026-09-05T11:59:23.269Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1131 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02H8BWEKY5SSC2PFADBN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.250" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.420565] gitaly[897]: time="2026-09-05T11:59:23.270Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1131 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02H8BWEKY5SSC2PFADBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.250" grpc.time_ms=19.431 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.462365] nixbot[3535]: INFO: ::ffff:127.0.0.1:59094 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.10 seconds) gitlab # [ 336.497910] gitaly[897]: time="2026-09-05T11:59:23.381Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02NSS4SKK93FD2PRFBGN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.372" grpc.time_ms=9.063 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.512517] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:23 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 61 gitlab # [ 336.582446] gitaly[897]: time="2026-09-05T11:59:23.464Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02QRC8SRTX87S1GCY7EZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.437" pid=897 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 336.608824] gitaly[897]: time="2026-09-05T11:59:23.465Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=23 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02QRC8SRTX87S1GCY7EZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.437" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.633475] gitlab-workhorse[823]: time="2026-09-05T11:59:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 336.636522] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 81 gitlab # [ 336.639558] gitaly[897]: time="2026-09-05T11:59:23.465Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=23 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02QRC8SRTX87S1GCY7EZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:23.437" grpc.time_ms=27.833 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 336.708660] gitaly[897]: time="2026-09-05T11:59:23.592Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T11:59:53.489" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T11:59:23.590" grpc.time_ms=1.083 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 336.766681] gitaly[897]: time="2026-09-05T11:59:23.649Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-05T11:59:53.533" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:23.633" grpc.time_ms=15.97 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 336.805918] gitaly[897]: time="2026-09-05T11:59:23.683Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T11:59:33.130" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:23.670" grpc.time_ms=12.349 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 336.830852] gitaly[897]: time="2026-09-05T11:59:23.708Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T11:59:33.155" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:23.696" grpc.time_ms=12.008 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 336.946696] gitaly[897]: time="2026-09-05T11:59:23.830Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-05T11:59:33.288" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:23.829" grpc.time_ms=0.62 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 336.981433] gitaly[897]: time="2026-09-05T11:59:23.864Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-05T11:59:33.006" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:23.847" grpc.time_ms=17.369 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 337.055213] gitaly[897]: time="2026-09-05T11:59:23.938Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:23.937" grpc.time_ms=0.944 pid=897 span.kind=server system=grpc gitlab # [ 337.103385] gitaly[897]: time="2026-09-05T11:59:23.986Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T11:59:23.984" grpc.time_ms=2.029 pid=897 span.kind=server system=grpc gitlab # [ 337.131446] gitaly[897]: time="2026-09-05T11:59:23.996Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 diskcache=2f5eadf4-dd8a-4172-a092-99c6c24e0b00 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-05T11:59:33.028" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:23.869" pid=897 ref=refs/keep-around/8a22cc6eb1d12352a83d313a21f80b5b21d790db remote_ip="2001:db8:1::1" revision=8a22cc6eb1d12352a83d313a21f80b5b21d790db span.kind=server system=grpc username=root gitlab # [ 337.154797] gitaly[897]: time="2026-09-05T11:59:24.001Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=124 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5574 command.oublock=8 command.real_time_ms=119 command.spawn_token_fork_ms=2 command.system_time_ms=75 command.user_time_ms=49 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-05T11:59:33.028" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T11:59:23.869" grpc.time_ms=127.263 pid=897 ref=refs/keep-around/8a22cc6eb1d12352a83d313a21f80b5b21d790db remote_ip="2001:db8:1::1" revision=8a22cc6eb1d12352a83d313a21f80b5b21d790db span.kind=server system=grpc username=root gitlab # [ 337.176886] gitaly[897]: time="2026-09-05T11:59:24.002Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T11:59:54.001" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T11:59:24.001" grpc.time_ms=0.561 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 337.339818] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 704 gitlab # [ 337.343112] nixbot[3535]: {"ts": "2026-09-05T11:59:24.224Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db \"HTTP/1.1 201 Created\""} gitlab # [ 337.361377] nixbot[3535]: {"ts": "2026-09-05T11:59:24.244Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-8a22cc6eb1d1-0"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.07 seconds) gitlab # [ 337.530481] nixbot[3535]: INFO: ::ffff:127.0.0.1:59104 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 337.658966] nixbot[3535]: {"ts": "2026-09-05T11:59:24.542Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-8a22cc6eb1d1-0", "argv0": "bwrap"} gitlab # [ 337.847738] systemd[1]: Started Nix Daemon. gitlab # [ 337.966789] nix-daemon[3765]: accepted connection from pid 3760, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 338.593972] nixbot[3535]: INFO: ::ffff:127.0.0.1:59112 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 339.649248] nixbot[3535]: INFO: ::ffff:127.0.0.1:59128 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 340.705998] nixbot[3535]: INFO: ::ffff:127.0.0.1:59132 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 341.761941] nixbot[3535]: INFO: ::ffff:127.0.0.1:50566 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 342.485816] nix-daemon[3765]: accepted connection from pid 3788, user nixbot gitlab # [ 342.509572] nix-daemon[3765]: accepted connection from pid 3760, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 342.819771] nixbot[3535]: INFO: ::ffff:127.0.0.1:50574 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 343.879996] nixbot[3535]: INFO: ::ffff:127.0.0.1:50582 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 344.942302] nixbot[3535]: INFO: ::ffff:127.0.0.1:50590 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 345.999127] nixbot[3535]: INFO: ::ffff:127.0.0.1:50602 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 346.912865] nix-daemon[3765]: accepted connection from pid 3760, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 347.057161] nixbot[3535]: INFO: ::ffff:127.0.0.1:50608 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 347.139189] gitaly[897]: time="2026-09-05T11:59:34.021Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0D001YPW4GBRE2XV3YVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:00:04.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T11:59:34.020" grpc.time_ms=1.299 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.221874] gitaly[897]: time="2026-09-05T11:59:34.105Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ0D001YPW4GBRE2XV3YVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T11:59:44.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:34.086" grpc.time_ms=17.915 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.330220] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 323 gitlab # [ 347.338346] nixbot[3535]: {"ts": "2026-09-05T11:59:34.215Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db \"HTTP/1.1 201 Created\""} gitlab # [ 347.431806] gitaly[897]: time="2026-09-05T11:59:34.315Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0DAEXM1GA1B8GP6TRS1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:00:04.312" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T11:59:34.313" grpc.time_ms=1.194 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.491817] gitaly[897]: time="2026-09-05T11:59:34.375Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ0DAEXM1GA1B8GP6TRS1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T11:59:44.277" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:34.357" grpc.time_ms=17.218 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.563837] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 224 gitlab # [ 347.573455] nixbot[3535]: {"ts": "2026-09-05T11:59:34.448Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db \"HTTP/1.1 201 Created\""} gitlab # [ 347.651049] gitaly[897]: time="2026-09-05T11:59:34.534Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0DJ3VSCKSJN7VVJ3Z3QR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:00:04.532" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T11:59:34.533" grpc.time_ms=1.288 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.699287] gitaly[897]: time="2026-09-05T11:59:34.582Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ0DJ3VSCKSJN7VVJ3Z3QR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T11:59:44.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T11:59:34.566" grpc.time_ms=15.993 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.761155] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 176 gitlab # [ 347.769069] nixbot[3535]: {"ts": "2026-09-05T11:59:34.645Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 348.126085] nixbot[3535]: INFO: ::ffff:127.0.0.1:50614 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 348.394451] gitaly[897]: time="2026-09-05T11:59:35.276Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ0E7NXCP53SS1M1BGB236 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.257" grpc.time_ms=19.518 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.434404] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:35 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 119 gitlab # [ 348.496173] gitaly[897]: time="2026-09-05T11:59:35.379Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0EBF096Y46PYNCMETJYZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.361" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 348.527331] gitaly[897]: time="2026-09-05T11:59:35.409Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0EBF096Y46PYNCMETJYZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.361" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.569624] gitaly[897]: time="2026-09-05T11:59:35.410Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0EBF096Y46PYNCMETJYZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.361" grpc.time_ms=48.683 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.599926] gitlab-workhorse[823]: time="2026-09-05T11:59:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 348.604815] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:35 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 131 gitlab # [ 348.711509] gitaly[897]: time="2026-09-05T11:59:35.594Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ0EJ95HJ1G0AVYRX8FAYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.582" grpc.time_ms=11.777 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.729349] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:35 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 97 gitlab # [ 348.807811] gitaly[897]: time="2026-09-05T11:59:35.691Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0ENDR8BKGXT0RVG7MRKK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.673" pid=897 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 348.838781] gitaly[897]: time="2026-09-05T11:59:35.691Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0ENDR8BKGXT0RVG7MRKK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.673" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.864965] gitlab-workhorse[823]: time="2026-09-05T11:59:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 348.867903] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:35 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 94 gitlab # [ 348.871167] gitaly[897]: time="2026-09-05T11:59:35.691Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0ENDR8BKGXT0RVG7MRKK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.673" grpc.time_ms=17.945 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.959151] gitaly[897]: time="2026-09-05T11:59:35.842Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ0ETDT3SFCQCHDGK4SVQP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.827" grpc.time_ms=15.065 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.997265] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:35 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 85 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 349.153972] gitaly[897]: time="2026-09-05T11:59:36.036Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0EY89SGF1GER3TSF6GKA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.999" pid=897 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 349.183509] gitaly[897]: time="2026-09-05T11:59:36.036Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=2 command.system_time_ms=23 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0EY89SGF1GER3TSF6GKA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.999" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.08 seconds) gitlab # [ 349.217430] gitlab-workhorse[823]: time="2026-09-05T11:59:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 349.222110] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:11:59:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 179 gitlab # [ 349.227146] nixbot[3535]: INFO: ::ffff:127.0.0.1:50620 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 349.231183] gitaly[897]: time="2026-09-05T11:59:36.037Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=2 command.system_time_ms=23 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ0EY89SGF1GER3TSF6GKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T11:59:35.999" grpc.time_ms=37.068 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 350.267900] nixbot[3535]: INFO: ::ffff:127.0.0.1:50634 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 351.321690] nixbot[3535]: INFO: ::ffff:127.0.0.1:37078 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 352.378926] nixbot[3535]: INFO: ::ffff:127.0.0.1:37090 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 353.434533] nixbot[3535]: INFO: ::ffff:127.0.0.1:37100 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 354.491607] nixbot[3535]: INFO: ::ffff:127.0.0.1:37102 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 355.547590] nixbot[3535]: INFO: ::ffff:127.0.0.1:37110 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 356.602507] nixbot[3535]: INFO: ::ffff:127.0.0.1:37112 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 357.658842] nixbot[3535]: INFO: ::ffff:127.0.0.1:37116 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 358.710738] nixbot[3535]: INFO: ::ffff:127.0.0.1:37126 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 359.766636] nixbot[3535]: INFO: ::ffff:127.0.0.1:37140 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 360.819553] nixbot[3535]: INFO: ::ffff:127.0.0.1:37146 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 361.875694] nixbot[3535]: INFO: ::ffff:127.0.0.1:55632 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 362.930259] nixbot[3535]: INFO: ::ffff:127.0.0.1:55644 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 363.984996] nixbot[3535]: INFO: ::ffff:127.0.0.1:55658 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 365.038177] nixbot[3535]: INFO: ::ffff:127.0.0.1:55660 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 366.092244] nixbot[3535]: INFO: ::ffff:127.0.0.1:55674 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 367.147754] nixbot[3535]: INFO: ::ffff:127.0.0.1:55684 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 368.201517] nixbot[3535]: INFO: ::ffff:127.0.0.1:55690 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 369.255349] nixbot[3535]: INFO: ::ffff:127.0.0.1:55694 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 370.312239] nixbot[3535]: INFO: ::ffff:127.0.0.1:55698 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 371.365094] nixbot[3535]: INFO: ::ffff:127.0.0.1:48458 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 371.986221] sidekiq-cluster[3481]: 2026-09-05T11:59:58.868Z pid=3481 tid=ylt INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3481, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433766400} gitlab # [ 372.277471] sidekiq-cluster[3481]: W, [2026-09-05T11:59:59.160737 #3481] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 372.342595] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T11:59:59.225Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 372.399395] nixbot[3535]: INFO: ::ffff:127.0.0.1:48472 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.04 seconds) gitlab # [ 373.118013] gitaly[897]: time="2026-09-05T12:00:00.000Z" level=info msg="maintenance: daily starting" max_duration=600000000000 pid=897 gitlab # [ 373.118409] gitaly[897]: time="2026-09-05T12:00:00.000Z" level=info msg="maintenance: optimizing repos in storage" pid=897 storage_path=/var/gitlab/state/repositories gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 373.424613] systemd[1]: Starting Logrotate Service... gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 373.458011] nixbot[3535]: INFO: ::ffff:127.0.0.1:48486 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 373.551587] systemd[1]: logrotate.service: Deactivated successfully. gitlab # [ 373.554698] systemd[1]: Finished Logrotate Service. gitlab # [ 374.127573] gitaly[897]: time="2026-09-05T12:00:01.009Z" level=info msg="repository info" pid=897 repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-05 11:59:14.34646326 +0000 UTC} {2 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" system=housekeeping gitlab # [ 374.171142] gitaly[897]: time="2026-09-05T12:00:01.054Z" level=info msg="removed files" pid=897 stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=housekeeping gitlab # [ 374.275510] gitaly[897]: time="2026-09-05T12:00:01.158Z" level=info msg="optimized repository" clean_stale_data_time=44.1713ms clean_work_trees_time="268.78µs" commit_graph_time=27.76502ms correlation_id= gl_project_path= gl_repo= optimizations="map[packed_objects_geometric:success written_bitmap:success written_commit_graph_full:success written_multi_pack_index:success]" pack_refs_time="2.44µs" pid=897 prune_time="3.16µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time=75.83124ms storage_name=default system=housekeeping total_time=148.26166ms gitlab # [ 374.296205] gitaly[897]: time="2026-09-05T12:00:01.159Z" level=info msg="maintenance: repo optimization succeeded" pid=897 relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git source=maintenance.daily start_time="2026-09-05 12:00:01.0007828 +0000 UTC" storage=default time_ms=158 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 374.516337] nixbot[3535]: INFO: ::ffff:127.0.0.1:48488 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 375.126441] gitaly[897]: time="2026-09-05T12:00:02.009Z" level=info msg="repository info" pid=897 repository_info="{false {0 0 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-05 11:59:14.5864831 +0000 UTC} {0 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" system=housekeeping gitlab # [ 375.173290] gitaly[897]: time="2026-09-05T12:00:02.055Z" level=info msg="removed files" pid=897 stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=housekeeping gitlab # [ 375.189780] gitaly[897]: time="2026-09-05T12:00:02.056Z" level=info msg="optimized repository" clean_stale_data_time=45.71392ms clean_work_trees_time="281.88µs" commit_graph_time="1.96µs" correlation_id= gl_project_path= gl_repo= optimizations="map[]" pack_refs_time="1µs" pid=897 prune_time=620ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git repack_time="3.04µs" storage_name=default system=housekeeping total_time=46.02384ms gitlab # [ 375.205160] gitaly[897]: time="2026-09-05T12:00:02.056Z" level=info msg="maintenance: repo optimization succeeded" pid=897 relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git source=maintenance.daily start_time="2026-09-05 12:00:02.00111236 +0000 UTC" storage=default time_ms=55 gitlab # [ 375.213554] gitaly[897]: time="2026-09-05T12:00:02.057Z" level=info msg="maintenance: daily completed" actual_duration=2.05756476s error="" max_duration=600000000000 pid=897 gitlab # [ 375.218000] gitaly[897]: time="2026-09-05T12:00:02.057Z" level=info msg="maintenance: daily scheduled" pid=897 scheduled="2026-09-06 12:00:00 +0000 UTC" gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 375.572790] nixbot[3535]: INFO: ::ffff:127.0.0.1:48490 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 376.028446] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:02.910Z","message":"GitLab reliable fetch activated!"} gitlab # [ 376.033313] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:02.910Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 376.627843] nixbot[3535]: INFO: ::ffff:127.0.0.1:48500 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 377.682660] nixbot[3535]: INFO: ::ffff:127.0.0.1:48508 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 378.736861] nixbot[3535]: INFO: ::ffff:127.0.0.1:48522 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 379.793160] nixbot[3535]: INFO: ::ffff:127.0.0.1:48528 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 380.849950] nixbot[3535]: INFO: ::ffff:127.0.0.1:56610 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 381.905783] nixbot[3535]: INFO: ::ffff:127.0.0.1:56626 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 382.962556] nixbot[3535]: INFO: ::ffff:127.0.0.1:56630 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 384.017741] nixbot[3535]: INFO: ::ffff:127.0.0.1:56632 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 385.073927] nixbot[3535]: INFO: ::ffff:127.0.0.1:56636 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 386.128790] nixbot[3535]: INFO: ::ffff:127.0.0.1:56644 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 387.183942] nixbot[3535]: INFO: ::ffff:127.0.0.1:56646 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 388.240223] nixbot[3535]: INFO: ::ffff:127.0.0.1:56654 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 389.296264] nixbot[3535]: INFO: ::ffff:127.0.0.1:56656 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 390.350462] nixbot[3535]: INFO: ::ffff:127.0.0.1:56668 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 391.407746] nixbot[3535]: INFO: ::ffff:127.0.0.1:47220 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 392.461157] nixbot[3535]: INFO: ::ffff:127.0.0.1:47228 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 393.515379] nixbot[3535]: INFO: ::ffff:127.0.0.1:47230 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 394.571677] nixbot[3535]: INFO: ::ffff:127.0.0.1:47232 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 395.625402] nixbot[3535]: INFO: ::ffff:127.0.0.1:47248 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 396.679511] nixbot[3535]: INFO: ::ffff:127.0.0.1:47260 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 397.733834] nixbot[3535]: INFO: ::ffff:127.0.0.1:47270 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 398.790996] nixbot[3535]: INFO: ::ffff:127.0.0.1:47276 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 399.847247] nixbot[3535]: INFO: ::ffff:127.0.0.1:47286 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 400.903409] nixbot[3535]: INFO: ::ffff:127.0.0.1:56480 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 401.959296] nixbot[3535]: INFO: ::ffff:127.0.0.1:56486 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 403.015892] nixbot[3535]: INFO: ::ffff:127.0.0.1:56500 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 404.071163] nixbot[3535]: INFO: ::ffff:127.0.0.1:56510 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 405.125191] nixbot[3535]: INFO: ::ffff:127.0.0.1:56518 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 406.178752] nixbot[3535]: INFO: ::ffff:127.0.0.1:56522 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 407.234133] nixbot[3535]: INFO: ::ffff:127.0.0.1:56532 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 408.288466] nixbot[3535]: INFO: ::ffff:127.0.0.1:56536 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 409.340564] nixbot[3535]: INFO: ::ffff:127.0.0.1:56540 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 410.395736] nixbot[3535]: INFO: ::ffff:127.0.0.1:56556 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 411.453000] nixbot[3535]: INFO: ::ffff:127.0.0.1:47058 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 412.507762] nixbot[3535]: INFO: ::ffff:127.0.0.1:47072 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 413.565451] nixbot[3535]: INFO: ::ffff:127.0.0.1:47084 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 413.909675] postgres[4138]: [4138] ERROR: relation "table_without_models" does not exist gitlab # [ 413.913269] postgres[4138]: [4138] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 414.620891] nixbot[3535]: INFO: ::ffff:127.0.0.1:47086 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 415.674333] nixbot[3535]: INFO: ::ffff:127.0.0.1:47088 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 416.730144] nixbot[3535]: INFO: ::ffff:127.0.0.1:47094 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 417.783210] nixbot[3535]: INFO: ::ffff:127.0.0.1:47100 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 418.807378] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:45.685Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 418.807602] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:45.688Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 418.808681] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:45.692Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 418.817689] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:45.699Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 418.823803] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:45.703Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 418.834985] nixbot[3535]: INFO: ::ffff:127.0.0.1:47114 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 419.891196] nixbot[3535]: INFO: ::ffff:127.0.0.1:47128 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 420.944450] nixbot[3535]: INFO: ::ffff:127.0.0.1:49354 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 421.464441] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:48.345Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"683928a7a8bc6a6fbb4e6dde","created_at":"2026-09-05T11:59:14.445Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:14.509Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-683928a7a8bc6a6fbb4e6dde: start","job_status":"start","queue_duration_s":93.835937,"scheduling_latency_s":93.836086} gitlab # [ 421.840224] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:48.722Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"683928a7a8bc6a6fbb4e6dde","created_at":"2026-09-05T11:59:14.445Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:14.509Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-683928a7a8bc6a6fbb4e6dde: done: 0.377216 sec","job_status":"done","queue_duration_s":93.835937,"scheduling_latency_s":93.836086,"redis_calls":23,"redis_duration_s":0.010106,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.004875,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00314,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002091,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.190617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19384376,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.377216,"completed_at":"2026-09-05T12:00:48.722Z","load_balancing_strategy":"primary","db_duration_s":0.10515,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 421.934696] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:48.725Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"98047e1923368faa914e20e6","created_at":"2026-09-05T11:59:14.528Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:14.538Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-98047e1923368faa914e20e6: start","job_status":"start","queue_duration_s":94.187455,"scheduling_latency_s":94.187491} gitlab # [ 422.000215] nixbot[3535]: INFO: ::ffff:127.0.0.1:49358 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 422.019542] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:48.902Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"98047e1923368faa914e20e6","created_at":"2026-09-05T11:59:14.528Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:14.538Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-98047e1923368faa914e20e6: done: 0.176853 sec","job_status":"done","queue_duration_s":94.187455,"scheduling_latency_s":94.187491,"redis_calls":14,"redis_duration_s":0.004572,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000928,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002119,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001525,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13759102,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.176853,"completed_at":"2026-09-05T12:00:48.902Z","load_balancing_strategy":"primary","db_duration_s":0.037102,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 422.107767] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:48.904Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"b394b9fbb63a230e1ee978bb","created_at":"2026-09-05T11:59:15.144Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:15.150Z","job_size_bytes":5,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-b394b9fbb63a230e1ee978bb: start","job_status":"start","queue_duration_s":93.753615,"scheduling_latency_s":93.753641} gitlab # [ 422.239105] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:49.122Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"b394b9fbb63a230e1ee978bb","created_at":"2026-09-05T11:59:15.144Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:15.150Z","job_size_bytes":5,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-b394b9fbb63a230e1ee978bb: done: 0.217605 sec","job_status":"done","queue_duration_s":93.753615,"scheduling_latency_s":93.753641,"redis_calls":12,"redis_duration_s":0.00326,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000852,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001911,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.169592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003225,"duration_s":0.217605,"completed_at":"2026-09-05T12:00:49.122Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0409,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 422.333986] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:49.123Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"b529369d2aabb409a9db59b7","created_at":"2026-09-05T11:59:15.185Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:15.191Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-b529369d2aabb409a9db59b7: start","job_status":"start","queue_duration_s":93.932654,"scheduling_latency_s":93.932672} gitlab # [ 422.558238] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:49.439Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"b529369d2aabb409a9db59b7","created_at":"2026-09-05T11:59:15.185Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:15.191Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-b529369d2aabb409a9db59b7: done: 0.315647 sec","job_status":"done","queue_duration_s":93.932654,"scheduling_latency_s":93.932672,"redis_calls":11,"redis_duration_s":0.003663,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000732,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002232,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000699,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.266,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.266,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.210193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00096732,"duration_s":0.315647,"completed_at":"2026-09-05T12:00:49.439Z","load_balancing_strategy":"primary","db_duration_s":0.107758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.664465] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:49.442Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"0c5674a44f3c8857cb49747a","created_at":"2026-09-05T11:59:15.194Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:15.200Z","job_size_bytes":89,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-0c5674a44f3c8857cb49747a: start","job_status":"start","queue_duration_s":94.242088,"scheduling_latency_s":94.242124} gitlab # [ 422.690480] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:49.514Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"0c5674a44f3c8857cb49747a","created_at":"2026-09-05T11:59:15.194Z","trace_propagation_headers":{"sentry-trace":"4b7301c4c5d94c8389f8fee15e09c436-02da17d671ec4211","baggage":"sentry-trace_id=4b7301c4c5d94c8389f8fee15e09c436,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZQSQPMMRAX3ZN4HV4MY9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:15.200Z","job_size_bytes":89,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-0c5674a44f3c8857cb49747a: done: 0.07246 sec","job_status":"done","queue_duration_s":94.242088,"scheduling_latency_s":94.242124,"redis_calls":11,"redis_duration_s":0.004204,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001136,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00232,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000748,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00078942,"duration_s":0.07246,"completed_at":"2026-09-05T12:00:49.514Z","load_balancing_strategy":"primary","db_duration_s":0.009548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.739097] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:49.518Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 8a22cc6eb1d12352a83d313a21f80b5b21d790db refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"44b29fd12438082c5a81d184","created_at":"2026-09-05T11:59:17.817Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-f32984e7747d4174","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:17.860Z","job_size_bytes":185,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-44b29fd12438082c5a81d184: start","job_status":"start","queue_duration_s":91.657448,"scheduling_latency_s":91.657482} gitlab # [ 422.755591] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:49.545Z","message":"changes: \"0000000000000000000000000000000000000000 8a22cc6eb1d12352a83d313a21f80b5b21d790db refs/heads/master\\n\""} gitlab # [ 422.758354] gitaly[897]: time="2026-09-05T12:00:49.601Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:00:59.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:49.586" grpc.time_ms=14.509 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.771034] gitaly[897]: time="2026-09-05T12:00:49.622Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-05T12:00:59.027" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:49.607" grpc.time_ms=14.085 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.817139] gitaly[897]: time="2026-09-05T12:00:49.700Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=716 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-05T12:00:59.106" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:49.686" grpc.time_ms=13.512 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.886811] gitaly[897]: time="2026-09-05T12:00:49.770Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:49.769" grpc.time_ms=0.705 pid=897 span.kind=server system=grpc gitlab # [ 422.940541] gitaly[897]: time="2026-09-05T12:00:49.823Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:49.823" grpc.time_ms=0.739 pid=897 span.kind=server system=grpc gitlab # [ 422.964493] gitaly[897]: time="2026-09-05T12:00:49.838Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F diskcache=4cf09c16-f896-491b-9a83-05554ec88e18 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-05T12:00:59.134" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:49.715" pid=897 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 422.984751] gitaly[897]: time="2026-09-05T12:00:49.838Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=133 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4982 command.oublock=8 command.real_time_ms=116 command.spawn_token_fork_ms=1 command.system_time_ms=73 command.user_time_ms=60 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-05T12:00:59.134" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:49.715" grpc.time_ms=123.376 pid=897 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 423.005498] gitaly[897]: time="2026-09-05T12:00:49.870Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-05T12:00:59.289" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:49.869" grpc.time_ms=0.351 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.018670] gitaly[897]: time="2026-09-05T12:00:49.902Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=859 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=3 command.system_time_ms=15 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-05T12:00:59.007" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:49.878" grpc.time_ms=23.69 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.07 seconds) gitlab # [ 423.071538] nixbot[3535]: INFO: ::ffff:127.0.0.1:49362 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 423.202694] gitaly[897]: time="2026-09-05T12:00:50.086Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=434 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-05T12:01:20.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:50.071" grpc.time_ms=13.909 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.261783] gitaly[897]: time="2026-09-05T12:00:50.145Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-05T12:01:00.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:50.144" grpc.time_ms=0.579 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.291880] gitaly[897]: time="2026-09-05T12:00:50.160Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=438 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-05T12:01:00.009" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-05T12:00:50.149" grpc.time_ms=10.808 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.318162] gitaly[897]: time="2026-09-05T12:00:50.201Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:00.043" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:50.183" grpc.time_ms=17.758 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.343555] gitaly[897]: time="2026-09-05T12:00:50.226Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:00.070" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:50.210" grpc.time_ms=15.89 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.398006] gitaly[897]: time="2026-09-05T12:00:50.281Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=2 command.system_time_ms=21 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-05T12:01:20.252" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:50.253" grpc.time_ms=28.042 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 424.043571] gitaly[897]: time="2026-09-05T12:00:50.925Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-05T12:01:00.053" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:50.923" grpc.time_ms=2.043 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.082011] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:50.964Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 8a22cc6eb1d12352a83d313a21f80b5b21d790db refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"44b29fd12438082c5a81d184","created_at":"2026-09-05T11:59:17.817Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-f32984e7747d4174","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:17.860Z","job_size_bytes":185,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-44b29fd12438082c5a81d184: done: 1.446507 sec","job_status":"done","queue_duration_s":91.657448,"scheduling_latency_s":91.657482,"gitaly_calls":13,"gitaly_duration_s":0.3357,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.067998,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.008567,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008731,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002189,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005651,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.027189,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.015671,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.092,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.323,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.373,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.092,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.953736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00094748,"exclusive_lock_hold_duration_s":0.03899292,"duration_s":1.446507,"completed_at":"2026-09-05T12:00:50.964Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.161504,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.07 seconds) gitlab # [ 424.132056] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:50.975Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","8a22cc6eb1d12352a83d313a21f80b5b21d790db 3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"08a130a12d2639b4926150b4","created_at":"2026-09-05T11:59:23.143Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-f39cacb068a345b7","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:23.146Z","job_size_bytes":185,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-08a130a12d2639b4926150b4: start","job_status":"start","queue_duration_s":87.828069,"scheduling_latency_s":87.828116} gitlab # [ 424.148642] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:51.007Z","message":"changes: \"8a22cc6eb1d12352a83d313a21f80b5b21d790db 3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 refs/heads/master\\n\""} gitlab # [ 424.151286] nixbot[3535]: INFO: ::ffff:127.0.0.1:49376 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 424.182693] gitaly[897]: time="2026-09-05T12:00:51.066Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:01.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:51.048" grpc.time_ms=17.099 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.207083] gitaly[897]: time="2026-09-05T12:00:51.090Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-05T12:01:01.033" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:51.074" grpc.time_ms=15.841 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.222505] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:51.099Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"10ad1539de104b0f8268c6d1","created_at":"2026-09-05T12:00:51.095Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-5fc97ef3ed2247c1","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"d4a5aaa29ecf472e3cc66828","job_size_bytes":46,"pid":3481,"job_status":"deduplicated","message":"ProjectCacheWorker JID-10ad1539de104b0f8268c6d1: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 424.258261] gitaly[897]: time="2026-09-05T12:00:51.141Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=716 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-05T12:01:01.084" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:51.124" grpc.time_ms=17.082 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.285391] gitaly[897]: time="2026-09-05T12:00:51.168Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=713 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-05T12:01:21.149" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:51.149" grpc.time_ms=18.874 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.323685] gitaly[897]: time="2026-09-05T12:00:51.200Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=440 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-05T12:01:01.049" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-05T12:00:51.190" grpc.time_ms=10.178 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.348550] gitaly[897]: time="2026-09-05T12:00:51.231Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:01.077" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:51.217" grpc.time_ms=14.084 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.638775] gitaly[897]: time="2026-09-05T12:00:51.522Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-05T12:01:01.243" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:51.383" grpc.time_ms=138.374 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.678070] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:51.546Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","8a22cc6eb1d12352a83d313a21f80b5b21d790db 3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"08a130a12d2639b4926150b4","created_at":"2026-09-05T11:59:23.143Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-f39cacb068a345b7","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T11:59:23.146Z","job_size_bytes":185,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-08a130a12d2639b4926150b4: done: 0.571333 sec","job_status":"done","queue_duration_s":87.828069,"scheduling_latency_s":87.828116,"gitaly_calls":7,"gitaly_duration_s":0.258268,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.035985,"redis_read_bytes":500,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.004134,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001582,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00709,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.011871,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.011308,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.039,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.039,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.401018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00154008,"exclusive_lock_hold_duration_s":0.03556638,"duration_s":0.571333,"completed_at":"2026-09-05T12:00:51.546Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043459,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 424.769484] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:51.547Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"6d9deef1993a1658c686303d","created_at":"2026-09-05T11:59:24.008Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-c5642477d0ea4ddc","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:24.017Z","job_size_bytes":65,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-6d9deef1993a1658c686303d: start","job_status":"start","queue_duration_s":87.530044,"scheduling_latency_s":87.530067} gitlab # [ 424.791567] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:51.654Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"6d9deef1993a1658c686303d","created_at":"2026-09-05T11:59:24.008Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-c5642477d0ea4ddc","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:24.017Z","job_size_bytes":65,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-6d9deef1993a1658c686303d: done: 0.107154 sec","job_status":"done","queue_duration_s":87.530044,"scheduling_latency_s":87.530067,"redis_calls":16,"redis_duration_s":0.004783,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002439,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005005,"duration_s":0.107154,"completed_at":"2026-09-05T12:00:51.654Z","load_balancing_strategy":"primary","db_duration_s":0.036465,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 424.838133] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:51.656Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c6351dba58eebf2dd8b224c3","created_at":"2026-09-05T11:59:24.018Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-c5642477d0ea4ddc","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:24.027Z","job_size_bytes":65,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c6351dba58eebf2dd8b224c3: start","job_status":"start","queue_duration_s":87.629246,"scheduling_latency_s":87.629267} gitlab # [ 424.966032] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:51.848Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c6351dba58eebf2dd8b224c3","created_at":"2026-09-05T11:59:24.018Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-c5642477d0ea4ddc","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:24.027Z","job_size_bytes":65,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c6351dba58eebf2dd8b224c3: done: 0.191652 sec","job_status":"done","queue_duration_s":87.629246,"scheduling_latency_s":87.629267,"redis_calls":21,"redis_duration_s":0.007217,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003256,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002077,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.125791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064648,"duration_s":0.191652,"completed_at":"2026-09-05T12:00:51.848Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.062897] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:51.851Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8ad740c1c8463ee9f17eb303","created_at":"2026-09-05T11:59:24.191Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-c5642477d0ea4ddc","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:24.199Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8ad740c1c8463ee9f17eb303: start","job_status":"start","queue_duration_s":87.651312,"scheduling_latency_s":87.651343} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.07 seconds) gitlab # [ 425.204682] nixbot[3535]: INFO: ::ffff:127.0.0.1:49386 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 425.275745] gitaly[897]: time="2026-09-05T12:00:52.157Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:52.156" grpc.time_ms=0.824 pid=897 span.kind=server system=grpc gitlab # [ 425.328228] gitaly[897]: time="2026-09-05T12:00:52.211Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:52.210" grpc.time_ms=0.66 pid=897 span.kind=server system=grpc gitlab # [ 425.383586] gitaly[897]: time="2026-09-05T12:00:52.266Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:52.266" grpc.time_ms=0.749 pid=897 span.kind=server system=grpc gitlab # [ 425.412457] gitaly[897]: time="2026-09-05T12:00:52.279Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 diskcache=92303721-1d70-403b-bd7a-1e0040caa673 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-05T12:01:22.099" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:52.100" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.433534] gitaly[897]: time="2026-09-05T12:00:52.279Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=201 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7547 command.oublock=0 command.real_time_ms=172 command.spawn_token_fork_ms=1 command.system_time_ms=93 command.user_time_ms=108 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-05T12:01:22.099" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:52.100" grpc.time_ms=179.464 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.590572] gitaly[897]: time="2026-09-05T12:00:52.473Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-05T12:01:22.472" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:52.472" grpc.time_ms=1.145 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.646591] gitaly[897]: time="2026-09-05T12:00:52.530Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ02VYQFF05W9R5RFF7MP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:22.528" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:52.528" grpc.time_ms=0.905 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.690847] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:52.572Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8ad740c1c8463ee9f17eb303","created_at":"2026-09-05T11:59:24.191Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-c5642477d0ea4ddc","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-05T11:59:24.199Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8ad740c1c8463ee9f17eb303: done: 0.72144 sec","job_status":"done","queue_duration_s":87.651312,"scheduling_latency_s":87.651343,"gitaly_calls":3,"gitaly_duration_s":0.192618,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.042885,"redis_read_bytes":457,"redis_write_bytes":27998,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001994,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002132,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.016477,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.005835,"redis_queues_read_bytes":36,"redis_queues_write_bytes":14493,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.00852,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001738,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00085,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003345,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.098,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.039,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.044,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.049,"db_ci_duration_s":0.049,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.398225,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.61181658,"duration_s":0.72144,"completed_at":"2026-09-05T12:00:52.572Z","load_balancing_strategy":"primary","db_duration_s":0.114077,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 425.757349] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:52.575Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"d4a5aaa29ecf472e3cc66828","created_at":"2026-09-05T12:00:49.626Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:49.649Z","job_size_bytes":46,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d4a5aaa29ecf472e3cc66828: start","job_status":"start","queue_duration_s":2.925808,"scheduling_latency_s":2.92585} gitlab # [ 425.782339] gitaly[897]: time="2026-09-05T12:00:52.665Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-05T18:00:52.658" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=55 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:52.659" grpc.time_ms=5.82 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.813518] gitaly[897]: time="2026-09-05T12:00:52.686Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=710 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-05T12:01:22.673" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:52.674" grpc.time_ms=12.014 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.849900] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:52.720Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"d4a5aaa29ecf472e3cc66828","created_at":"2026-09-05T12:00:49.626Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:49.649Z","job_size_bytes":46,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d4a5aaa29ecf472e3cc66828: done: 0.14465 sec","job_status":"done","queue_duration_s":2.925808,"scheduling_latency_s":2.92585,"gitaly_calls":2,"gitaly_duration_s":0.023854,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008971,"redis_read_bytes":69,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001651,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000599,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002078,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003643,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":751,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003555,"duration_s":0.14465,"completed_at":"2026-09-05T12:00:52.720Z","load_balancing_strategy":"primary","db_duration_s":0.017438,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 425.923774] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:52.722Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0126af5e6170e005c8451b3d","created_at":"2026-09-05T12:00:49.673Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:49.680Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0126af5e6170e005c8451b3d: start","job_status":"start","queue_duration_s":3.04135,"scheduling_latency_s":3.041378} gitlab # [ 425.946509] gitaly[897]: time="2026-09-05T12:00:52.780Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F diskcache=8e287b19-5cb1-4162-b64f-2efd8ac1911f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-05T18:00:52.748" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:52.749" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.960549] gitaly[897]: time="2026-09-05T12:00:52.781Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1301 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=3 command.system_time_ms=19 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-05T18:00:52.748" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:52.749" grpc.time_ms=31.568 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.978640] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:52.831Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0126af5e6170e005c8451b3d","created_at":"2026-09-05T12:00:49.673Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:49.680Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0126af5e6170e005c8451b3d: done: 0.108992 sec","job_status":"done","queue_duration_s":3.04135,"scheduling_latency_s":3.041378,"gitaly_calls":1,"gitaly_duration_s":0.03498,"redis_calls":15,"redis_duration_s":0.004129,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001602,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000592,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001496,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.027,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08213551999999999,"duration_s":0.108992,"completed_at":"2026-09-05T12:00:52.831Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.026125] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:52.833Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"ba5d8ce7f8d003e1b4256977","created_at":"2026-09-05T12:00:49.860Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:49.864Z","job_size_bytes":23,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ba5d8ce7f8d003e1b4256977: start","job_status":"start","queue_duration_s":2.968493,"scheduling_latency_s":2.968516} gitlab # [ 426.041694] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:52.896Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"ba5d8ce7f8d003e1b4256977","created_at":"2026-09-05T12:00:49.860Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:49.864Z","job_size_bytes":23,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ba5d8ce7f8d003e1b4256977: done: 0.063081 sec","job_status":"done","queue_duration_s":2.968493,"scheduling_latency_s":2.968516,"gitaly_calls":1,"gitaly_duration_s":0.019807,"redis_calls":16,"redis_duration_s":0.00515,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000326,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002329,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063081,"completed_at":"2026-09-05T12:00:52.896Z","load_balancing_strategy":"primary","db_duration_s":0.004382,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 426.086570] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:52.898Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"50940d178d48c526895b16d4","created_at":"2026-09-05T12:00:50.698Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6b65ce10d4633cb6ffe54968383edffdfb120ad3a2f3b936e92d09b410943b01","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:50.706Z","job_size_bytes":467,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-50940d178d48c526895b16d4: start","job_status":"start","queue_duration_s":2.191108,"scheduling_latency_s":2.19113} gitlab # [ 426.103732] gitaly[897]: time="2026-09-05T12:00:52.885Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=710 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-05T12:01:22.867" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:52.867" grpc.time_ms=17.139 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.117177] gitaly[897]: time="2026-09-05T12:00:52.928Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:22.027" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:52.928" grpc.time_ms=0.372 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.129557] gitaly[897]: time="2026-09-05T12:00:52.940Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-05T12:01:02.002" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:52.933" grpc.time_ms=7.67 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.186717] gitaly[897]: time="2026-09-05T12:00:53.070Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-05T12:01:23.068" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:53.069" grpc.time_ms=0.946 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 426.199324] gitaly[897]: time="2026-09-05T12:00:53.081Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=28 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-05T12:01:23.077" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:53.077" grpc.time_ms=3.061 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.07 seconds) gitlab # [ 426.271118] nixbot[3535]: INFO: ::ffff:127.0.0.1:49390 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 426.321595] gitaly[897]: time="2026-09-05T12:00:53.203Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:03.146" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:53.186" grpc.time_ms=16.773 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.972423] gitaly[897]: time="2026-09-05T12:00:53.855Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=3 command.system_time_ms=16 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-05T12:01:23.828" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:53.829" grpc.time_ms=26.464 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.999926] gitaly[897]: time="2026-09-05T12:00:53.883Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RPZW09FP863AAXMJ1HX43F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-05T12:01:23.861" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:53.861" grpc.time_ms=21.679 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.042121] sidekiq-cluster[3481]: {"severity":"WARN","time":"2026-09-05T12:00:53.924Z","class":"CreatePipelineWorker","correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"8a22cc6eb1d12352a83d313a21f80b5b21d790db","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"8a22cc6eb1d12352a83d313a21f80b5b21d790db","ref":"refs/heads/master"},"retry":0} gitlab # [ 427.095288] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:53.934Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"50940d178d48c526895b16d4","created_at":"2026-09-05T12:00:50.698Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6b65ce10d4633cb6ffe54968383edffdfb120ad3a2f3b936e92d09b410943b01","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:50.706Z","job_size_bytes":467,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-50940d178d48c526895b16d4: done: 1.036701 sec","job_status":"done","queue_duration_s":2.191108,"scheduling_latency_s":2.19113,"gitaly_calls":7,"gitaly_duration_s":0.100193,"redis_calls":51,"redis_duration_s":0.013963,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.001544,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005697,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001815,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003524,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001383,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.865738,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058734,"duration_s":1.036701,"completed_at":"2026-09-05T12:00:53.934Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.080129,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 427.167426] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:53.937Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e2d6e56c-872e-4d85-9cbe-55597629df1d\"}"],"class":"WebHookWorker","jid":"0dde78e76eec392657f895d6","created_at":"2026-09-05T12:00:50.882Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ab90b4e62d433ed75383853bb6ff6d460882892410ef50ef6f83fe35a5aed1ed","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:50.889Z","job_size_bytes":1751,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0dde78e76eec392657f895d6: start","job_status":"start","queue_duration_s":3.047919,"scheduling_latency_s":3.047947} gitlab # [ 427.239509] nixbot[3535]: INFO: ::ffff:127.0.0.1:49402 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.06 seconds) gitlab # [ 427.279118] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:54.161Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e2d6e56c-872e-4d85-9cbe-55597629df1d\"}"],"class":"WebHookWorker","jid":"0dde78e76eec392657f895d6","created_at":"2026-09-05T12:00:50.882Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-699dd9ebd3ba429b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ab90b4e62d433ed75383853bb6ff6d460882892410ef50ef6f83fe35a5aed1ed","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:50.889Z","job_size_bytes":1751,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0dde78e76eec392657f895d6: done: 0.224338 sec","job_status":"done","queue_duration_s":3.047919,"scheduling_latency_s":3.047947,"redis_calls":19,"redis_duration_s":0.00685,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001631,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000709,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002968,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001542,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.09095174000003681,"cpu_s":0.112245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00086314,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.224338,"completed_at":"2026-09-05T12:00:54.161Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.392116] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:54.164Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"d7cc19b744f8c899b68b7a41","created_at":"2026-09-05T12:00:51.111Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-5fc97ef3ed2247c1","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"0126af5e6170e005c8451b3d","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:51.116Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-d7cc19b744f8c899b68b7a41: start","job_status":"start","queue_duration_s":3.047591,"scheduling_latency_s":3.047629} gitlab # [ 427.418588] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:54.268Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"d7cc19b744f8c899b68b7a41","created_at":"2026-09-05T12:00:51.111Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-5fc97ef3ed2247c1","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"0126af5e6170e005c8451b3d","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:51.116Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-d7cc19b744f8c899b68b7a41: done: 0.103984 sec","job_status":"done","queue_duration_s":3.047591,"scheduling_latency_s":3.047629,"gitaly_calls":1,"gitaly_duration_s":0.019415,"redis_calls":12,"redis_duration_s":0.009682,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002274,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05249554,"duration_s":0.103984,"completed_at":"2026-09-05T12:00:54.268Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.471065] gitaly[897]: time="2026-09-05T12:00:54.234Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 diskcache=b9eb58ae-740c-460d-8d6e-32e8624a23bb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-05T18:00:54.219" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:54.220" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.480636] gitaly[897]: time="2026-09-05T12:00:54.235Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-05T18:00:54.219" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:54.220" grpc.time_ms=14.538 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.493796] nixbot[3535]: INFO: ::ffff:127.0.0.1:49410 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 427.495998] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 125 gitlab # [ 427.498059] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:54.271Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1173e8f94ac593197deb5071","created_at":"2026-09-05T12:00:51.319Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-5fc97ef3ed2247c1","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:51738f912f9f9388a4f69fdadc0d16ed5a2aac5faf35a65cd04b6d0f2e55bc00","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:51.325Z","job_size_bytes":467,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1173e8f94ac593197deb5071: start","job_status":"start","queue_duration_s":2.946117,"scheduling_latency_s":2.946148} gitlab # [ 427.514832] gitaly[897]: time="2026-09-05T12:00:54.298Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=10 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2VCXJCJZBH780PS6P73K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:54.278" grpc.time_ms=19.761 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 427.527767] gitaly[897]: time="2026-09-05T12:00:54.316Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:24.315" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:54.315" grpc.time_ms=0.595 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.539889] gitaly[897]: time="2026-09-05T12:00:54.331Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-05T12:01:04.279" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:54.320" grpc.time_ms=11.519 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.552467] gitaly[897]: time="2026-09-05T12:00:54.392Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-05T12:01:24.390" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:54.391" grpc.time_ms=1.448 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.564810] gitaly[897]: time="2026-09-05T12:00:54.400Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=17 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-05T12:01:24.397" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:54.398" grpc.time_ms=2.522 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.580398] gitaly[897]: time="2026-09-05T12:00:54.453Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:04.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:54.437" grpc.time_ms=15.518 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.598998] gitaly[897]: time="2026-09-05T12:00:54.482Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2VGZDFQ1MYB4RYFBTT9V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:54.468" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 427.609120] gitaly[897]: time="2026-09-05T12:00:54.482Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2VGZDFQ1MYB4RYFBTT9V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:54.468" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 427.622011] gitlab-workhorse[823]: time="2026-09-05T12:00:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 427.623800] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:54 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 180 gitlab # [ 427.625706] gitaly[897]: time="2026-09-05T12:00:54.483Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2VGZDFQ1MYB4RYFBTT9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:54.468" grpc.time_ms=14.545 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 427.657759] nixbot[3535]: {"ts": "2026-09-05T12:00:54.541Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 427.801899] gitaly[897]: time="2026-09-05T12:00:54.685Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:24.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:54.684" grpc.time_ms=0.782 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.113780] gitaly[897]: time="2026-09-05T12:00:54.996Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:04.078" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:54.978" grpc.time_ms=18.02 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.254090] gitaly[897]: time="2026-09-05T12:00:55.137Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=17 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-05T12:01:25.110" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:55.110" grpc.time_ms=26.054 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.294182] gitaly[897]: time="2026-09-05T12:00:55.158Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ01TB7G690X5WZA7AJ5F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-05T12:01:25.142" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:55.142" grpc.time_ms=15.98 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.318307] sidekiq-cluster[3481]: {"severity":"WARN","time":"2026-09-05T12:00:55.172Z","class":"CreatePipelineWorker","correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"8a22cc6eb1d12352a83d313a21f80b5b21d790db","after":"3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1","ref":"refs/heads/master"},"retry":0} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.05 seconds) gitlab # [ 428.328748] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.178Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1173e8f94ac593197deb5071","created_at":"2026-09-05T12:00:51.319Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-5fc97ef3ed2247c1","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:51738f912f9f9388a4f69fdadc0d16ed5a2aac5faf35a65cd04b6d0f2e55bc00","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:51.325Z","job_size_bytes":467,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1173e8f94ac593197deb5071: done: 0.906685 sec","job_status":"done","queue_duration_s":2.946117,"scheduling_latency_s":2.946148,"gitaly_calls":7,"gitaly_duration_s":0.095853,"redis_calls":33,"redis_duration_s":0.00634,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.00103,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003203,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.782685,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.906685,"completed_at":"2026-09-05T12:00:55.178Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032187,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.390864] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 583 gitlab # [ 428.394004] nixbot[3535]: {"ts": "2026-09-05T12:00:55.154Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db \"HTTP/1.1 201 Created\""} gitlab # [ 428.397341] nixbot[3535]: INFO: ::ffff:127.0.0.1:49416 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 428.398829] nixbot[3535]: INFO: ::ffff:127.0.0.1:49432 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 428.401321] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.180Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"85ead528-58ff-44df-aa3e-95fb16f3dd1f\"}"],"class":"WebHookWorker","jid":"4cb30c3895b45723f4bb34b1","created_at":"2026-09-05T12:00:51.355Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-5fc97ef3ed2247c1","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1263d165ccfe3385037f7f8f3255650a24d80f57890eb6f183cba6ad16f3e917","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:51.359Z","job_size_bytes":1749,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4cb30c3895b45723f4bb34b1: start","job_status":"start","queue_duration_s":3.821766,"scheduling_latency_s":3.8218} gitlab # [ 428.420783] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.226Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"85ead528-58ff-44df-aa3e-95fb16f3dd1f\"}"],"class":"WebHookWorker","jid":"4cb30c3895b45723f4bb34b1","created_at":"2026-09-05T12:00:51.355Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-5fc97ef3ed2247c1","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1263d165ccfe3385037f7f8f3255650a24d80f57890eb6f183cba6ad16f3e917","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:51.359Z","job_size_bytes":1749,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4cb30c3895b45723f4bb34b1: done: 0.045605 sec","job_status":"done","queue_duration_s":3.821766,"scheduling_latency_s":3.8218,"redis_calls":14,"redis_duration_s":0.00315,"redis_read_bytes":26,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000426,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001973,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000751,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008046639999975014,"cpu_s":0.027117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.045605,"completed_at":"2026-09-05T12:00:55.226Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.469416] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.227Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f614b895240ea629ad5ef981","created_at":"2026-09-05T12:00:52.059Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.069Z","job_size_bytes":23,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f614b895240ea629ad5ef981: start","job_status":"start","queue_duration_s":3.158492,"scheduling_latency_s":3.15851} gitlab # [ 428.486010] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.350Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f614b895240ea629ad5ef981","created_at":"2026-09-05T12:00:52.059Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.069Z","job_size_bytes":23,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f614b895240ea629ad5ef981: done: 0.122411 sec","job_status":"done","queue_duration_s":3.158492,"scheduling_latency_s":3.15851,"redis_calls":10,"redis_duration_s":0.003617,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001515,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001541,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000561,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00073808,"duration_s":0.122411,"completed_at":"2026-09-05T12:00:55.350Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031289,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.533409] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:55 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 63 gitlab # [ 428.535474] gitlab-workhorse[823]: time="2026-09-05T12:00:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 428.537418] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 64 gitlab # [ 428.539610] gitaly[897]: time="2026-09-05T12:00:55.285Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WBTSDVS4666VCHMMXRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:25.282" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:55.283" grpc.time_ms=1.322 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.552638] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.353Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"acc50ff366e15598e5a6165b","created_at":"2026-09-05T12:00:52.289Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.303Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-acc50ff366e15598e5a6165b: start","job_status":"start","queue_duration_s":3.050334,"scheduling_latency_s":3.050359} gitlab # [ 428.569303] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.394Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"acc50ff366e15598e5a6165b","created_at":"2026-09-05T12:00:52.289Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.303Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-acc50ff366e15598e5a6165b: done: 0.040958 sec","job_status":"done","queue_duration_s":3.050334,"scheduling_latency_s":3.050359,"redis_calls":10,"redis_duration_s":0.001768,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031562,"duration_s":0.040958,"completed_at":"2026-09-05T12:00:55.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005773,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.614060] gitaly[897]: time="2026-09-05T12:00:55.302Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WE78AG15MVJCPWXSWRB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:55.294" grpc.time_ms=8.338 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 428.627110] gitaly[897]: time="2026-09-05T12:00:55.369Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WG8369EFSXRRZGZ13VM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:55.359" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 428.638500] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 333 gitlab # [ 428.641053] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:55 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 96 gitlab # [ 428.643131] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.398Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c4285085013e873d730f16d2","created_at":"2026-09-05T12:00:52.328Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.343Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c4285085013e873d730f16d2: start","job_status":"start","queue_duration_s":3.055816,"scheduling_latency_s":3.055837} gitlab # [ 428.659137] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.445Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c4285085013e873d730f16d2","created_at":"2026-09-05T12:00:52.328Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.343Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c4285085013e873d730f16d2: done: 0.046661 sec","job_status":"done","queue_duration_s":3.055816,"scheduling_latency_s":3.055837,"redis_calls":10,"redis_duration_s":0.0025770000000000003,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000334,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043686,"duration_s":0.046661,"completed_at":"2026-09-05T12:00:55.445Z","load_balancing_strategy":"primary","db_duration_s":0.006347,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.702878] gitaly[897]: time="2026-09-05T12:00:55.369Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WG8369EFSXRRZGZ13VM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:55.359" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 428.716568] nixbot[3535]: {"ts": "2026-09-05T12:00:55.497Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db \"HTTP/1.1 201 Created\""} gitlab # [ 428.719231] gitlab-workhorse[823]: time="2026-09-05T12:00:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 428.721140] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 65 gitlab # [ 428.723019] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.446Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"04e0314b996a20a6e302a996","created_at":"2026-09-05T12:00:52.368Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.378Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-04e0314b996a20a6e302a996: start","job_status":"start","queue_duration_s":3.06837,"scheduling_latency_s":3.068388} gitlab # [ 428.738735] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.492Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"04e0314b996a20a6e302a996","created_at":"2026-09-05T12:00:52.368Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.378Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-04e0314b996a20a6e302a996: done: 0.045934 sec","job_status":"done","queue_duration_s":3.06837,"scheduling_latency_s":3.068388,"redis_calls":13,"redis_duration_s":0.004046,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002832,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000366,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000576,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049566,"duration_s":0.045934,"completed_at":"2026-09-05T12:00:55.492Z","load_balancing_strategy":"primary","db_duration_s":0.008911,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.784294] gitaly[897]: time="2026-09-05T12:00:55.369Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WG8369EFSXRRZGZ13VM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:55.359" grpc.time_ms=9.801 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 428.798808] nixbot[3535]: {"ts": "2026-09-05T12:00:55.666Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db \"HTTP/1.1 201 Created\""} gitlab # [ 428.801644] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.494Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"9c7cdb7a081afd20a115d949","created_at":"2026-09-05T12:00:52.379Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.387Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-9c7cdb7a081afd20a115d949: start","job_status":"start","queue_duration_s":3.106446,"scheduling_latency_s":3.106466} gitlab # [ 428.818131] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.532Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"9c7cdb7a081afd20a115d949","created_at":"2026-09-05T12:00:52.379Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.387Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-9c7cdb7a081afd20a115d949: done: 0.037683 sec","job_status":"done","queue_duration_s":3.106446,"scheduling_latency_s":3.106466,"redis_calls":10,"redis_duration_s":0.001421,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000284,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025504,"duration_s":0.037683,"completed_at":"2026-09-05T12:00:55.532Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.864283] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8a22cc6eb1d12352a83d313a21f80b5b21d790db HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 162 gitlab # [ 428.866693] gitaly[897]: time="2026-09-05T12:00:55.409Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WBTSDVS4666VCHMMXRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:05.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:55.393" grpc.time_ms=16.21 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.880133] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a36007b9417ca8c1edfe284b","created_at":"2026-09-05T12:00:52.388Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.398Z","job_size_bytes":19,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a36007b9417ca8c1edfe284b: start","job_status":"start","queue_duration_s":3.137512,"scheduling_latency_s":3.137532} gitlab # [ 428.896287] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.570Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a36007b9417ca8c1edfe284b","created_at":"2026-09-05T12:00:52.388Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.398Z","job_size_bytes":19,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a36007b9417ca8c1edfe284b: done: 0.034223 sec","job_status":"done","queue_duration_s":3.137512,"scheduling_latency_s":3.137532,"redis_calls":11,"redis_duration_s":0.002892,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002267,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024958,"duration_s":0.034223,"completed_at":"2026-09-05T12:00:55.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0039,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.940822] gitaly[897]: time="2026-09-05T12:00:55.504Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WKKHB9RCCTPJYEBSZV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:55.493" grpc.time_ms=10.963 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 428.953994] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.572Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9f86d24deaac1c6f90c0fb39","created_at":"2026-09-05T12:00:52.402Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.409Z","job_size_bytes":66,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9f86d24deaac1c6f90c0fb39: start","job_status":"start","queue_duration_s":3.162099,"scheduling_latency_s":3.162119} gitlab # [ 428.971803] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.611Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9f86d24deaac1c6f90c0fb39","created_at":"2026-09-05T12:00:52.402Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.409Z","job_size_bytes":66,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9f86d24deaac1c6f90c0fb39: done: 0.039094 sec","job_status":"done","queue_duration_s":3.162099,"scheduling_latency_s":3.162119,"redis_calls":10,"redis_duration_s":0.003574,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000985,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00107894,"duration_s":0.039094,"completed_at":"2026-09-05T12:00:55.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.017592] gitaly[897]: time="2026-09-05T12:00:55.561Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WPFY5P47KEEE2YHQ64A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:25.056" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:55.556" grpc.time_ms=4.499 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.030569] gitaly[897]: time="2026-09-05T12:00:55.573Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WPNTS86F0WD71Z55YNQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:55.563" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 429.040848] nixbot[3535]: {"ts": "2026-09-05T12:00:55.890Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 \"HTTP/1.1 201 Created\""} gitlab # [ 429.043547] nixbot[3535]: {"ts": "2026-09-05T12:00:55.897Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-3dfbb554ce9a-5"} gitlab # [ 429.047049] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.612Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"33ec3dffd4a10acfe6463d59","created_at":"2026-09-05T12:00:52.411Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.417Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-33ec3dffd4a10acfe6463d59: start","job_status":"start","queue_duration_s":3.194941,"scheduling_latency_s":3.194961} gitlab # [ 429.063215] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.666Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"33ec3dffd4a10acfe6463d59","created_at":"2026-09-05T12:00:52.411Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.417Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-33ec3dffd4a10acfe6463d59: done: 0.053664 sec","job_status":"done","queue_duration_s":3.194941,"scheduling_latency_s":3.194961,"redis_calls":10,"redis_duration_s":0.007502999999999999,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000848,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005362,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025294,"duration_s":0.053664,"completed_at":"2026-09-05T12:00:55.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009325,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.107525] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 274 gitlab # [ 429.110209] gitaly[897]: time="2026-09-05T12:00:55.573Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WPNTS86F0WD71Z55YNQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:55.563" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.123609] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.667Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"3cbcd20d7f87d4b1d2c8eea8","created_at":"2026-09-05T12:00:52.427Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.433Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3cbcd20d7f87d4b1d2c8eea8: start","job_status":"start","queue_duration_s":3.233479,"scheduling_latency_s":3.233497} gitlab # [ 429.140113] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.707Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"3cbcd20d7f87d4b1d2c8eea8","created_at":"2026-09-05T12:00:52.427Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.433Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3cbcd20d7f87d4b1d2c8eea8: done: 0.039716 sec","job_status":"done","queue_duration_s":3.233479,"scheduling_latency_s":3.233497,"redis_calls":10,"redis_duration_s":0.00245,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000623,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001543,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020352,"duration_s":0.039716,"completed_at":"2026-09-05T12:00:55.707Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.184130] gitaly[897]: time="2026-09-05T12:00:55.574Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WPNTS86F0WD71Z55YNQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:00:55.563" grpc.time_ms=10.189 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.198416] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.708Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"90552977fff0c2dcd9070e91","created_at":"2026-09-05T12:00:52.434Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.439Z","job_size_bytes":6,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-90552977fff0c2dcd9070e91: start","job_status":"start","queue_duration_s":3.268388,"scheduling_latency_s":3.268404} gitlab # [ 429.214187] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.735Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"90552977fff0c2dcd9070e91","created_at":"2026-09-05T12:00:52.434Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.439Z","job_size_bytes":6,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-90552977fff0c2dcd9070e91: done: 0.026814 sec","job_status":"done","queue_duration_s":3.268388,"scheduling_latency_s":3.268404,"redis_calls":10,"redis_duration_s":0.002192,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000357,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001507,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021146,"duration_s":0.026814,"completed_at":"2026-09-05T12:00:55.735Z","load_balancing_strategy":"primary","db_duration_s":0.003254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.258819] nixbot[3535]: {"ts": "2026-09-05T12:00:56.030Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-3dfbb554ce9a-5", "argv0": "bwrap"} gitlab # [ 429.261264] gitaly[897]: time="2026-09-05T12:00:55.611Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WPFY5P47KEEE2YHQ64A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:05.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:55.603" grpc.time_ms=7.774 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.274866] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.736Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6179ae03d5ed7213858625eb","created_at":"2026-09-05T12:00:52.440Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.445Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6179ae03d5ed7213858625eb: start","job_status":"start","queue_duration_s":3.290637,"scheduling_latency_s":3.290654} gitlab # [ 429.290234] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.776Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6179ae03d5ed7213858625eb","created_at":"2026-09-05T12:00:52.440Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.445Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6179ae03d5ed7213858625eb: done: 0.04055 sec","job_status":"done","queue_duration_s":3.290637,"scheduling_latency_s":3.290654,"redis_calls":10,"redis_duration_s":0.004485,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002932,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000566,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023508,"duration_s":0.04055,"completed_at":"2026-09-05T12:00:55.776Z","load_balancing_strategy":"primary","db_duration_s":0.007709,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.333167] gitaly[897]: time="2026-09-05T12:00:55.671Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:25.670" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:55.670" grpc.time_ms=0.388 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.346007] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.778Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"9398576e5aad71afbaee2c8f","created_at":"2026-09-05T12:00:52.453Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.458Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-9398576e5aad71afbaee2c8f: start","job_status":"start","queue_duration_s":3.319266,"scheduling_latency_s":3.319284} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1' gitlab # [ 429.361560] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.851Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"9398576e5aad71afbaee2c8f","created_at":"2026-09-05T12:00:52.453Z","trace_propagation_headers":{"sentry-trace":"79ee8cfea3814ba8aeb11ffe85d8ec88-cb9d1794c2244733","baggage":"sentry-trace_id=79ee8cfea3814ba8aeb11ffe85d8ec88,sentry-release=Unknown"},"correlation_id":"01M1RQ02VYQFF05W9R5RFF7MP9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.458Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-9398576e5aad71afbaee2c8f: done: 0.073205 sec","job_status":"done","queue_duration_s":3.319266,"scheduling_latency_s":3.319284,"redis_calls":14,"redis_duration_s":0.0076040000000000005,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000288,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002177,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005139,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00109388,"duration_s":0.073205,"completed_at":"2026-09-05T12:00:55.851Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.403865] gitaly[897]: time="2026-09-05T12:00:55.688Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=718 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-05T12:01:25.679" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:55.680" grpc.time_ms=7.883 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.416751] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.853Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"4465c05a56775f8cd9c6a738","created_at":"2026-09-05T12:00:52.699Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-e4e6d60def6b437b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.704Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-4465c05a56775f8cd9c6a738: start","job_status":"start","queue_duration_s":3.148504,"scheduling_latency_s":3.148536} gitlab # [ 429.432336] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.921Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"4465c05a56775f8cd9c6a738","created_at":"2026-09-05T12:00:52.699Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-e4e6d60def6b437b","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:52.704Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-4465c05a56775f8cd9c6a738: done: 0.067789 sec","job_status":"done","queue_duration_s":3.148504,"scheduling_latency_s":3.148536,"redis_calls":19,"redis_duration_s":0.004111,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000745,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000607,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001883,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000876,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01113184,"duration_s":0.067789,"completed_at":"2026-09-05T12:00:55.921Z","load_balancing_strategy":"primary","db_duration_s":0.007254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.478151] gitaly[897]: time="2026-09-05T12:00:55.700Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:05.133" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:55.694" grpc.time_ms=6.562 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.491283] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.923Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0dde78e76eec392657f895d6"],"class":"WebHooks::LogExecutionWorker","jid":"cd321f4210e2bce827805d41","created_at":"2026-09-05T12:00:54.128Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-93f660c47e854b8c","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f4dccf0045ef4d9c024c836b52df0f10e1f05982d4d07a6367c82a5b221d23a4","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:54.141Z","job_size_bytes":2593,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-cd321f4210e2bce827805d41: start","job_status":"start","queue_duration_s":1.781138,"scheduling_latency_s":1.781159} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', in 0.13 seconds) (finished: subtest: push delivers the webhook, in 94.26 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 429.507982] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.992Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0dde78e76eec392657f895d6"],"class":"WebHooks::LogExecutionWorker","jid":"cd321f4210e2bce827805d41","created_at":"2026-09-05T12:00:54.128Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-93f660c47e854b8c","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f4dccf0045ef4d9c024c836b52df0f10e1f05982d4d07a6367c82a5b221d23a4","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:54.141Z","job_size_bytes":2593,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-cd321f4210e2bce827805d41: done: 0.069821 sec","job_status":"done","queue_duration_s":1.781138,"scheduling_latency_s":1.781159,"redis_calls":10,"redis_duration_s":0.002337,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000519,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051474,"duration_s":0.069821,"completed_at":"2026-09-05T12:00:55.992Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014976,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.551542] gitaly[897]: time="2026-09-05T12:00:55.718Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-05T12:01:05.157" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:55.718" grpc.time_ms=0.242 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.562986] nix-daemon[3765]: accepted connection from pid 4472, user nixbot gitlab # [ 429.563936] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:55.994Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"dbe1c9f856e4076f3eeb16bf","created_at":"2026-09-05T12:00:55.118Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-c2ac31ef729f40a2","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.125Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-dbe1c9f856e4076f3eeb16bf: start","job_status":"start","queue_duration_s":0.869206,"scheduling_latency_s":0.869225} gitlab # [ 429.579479] nixbot[3535]: INFO: ::ffff:127.0.0.1:49436 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 200 OK gitlab # [ 429.581454] gitaly[897]: time="2026-09-05T12:00:55.730Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-05T12:01:05.163" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:55.724" grpc.time_ms=6.717 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.594441] gitaly[897]: time="2026-09-05T12:00:55.765Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:55.765" grpc.time_ms=0.385 pid=897 span.kind=server system=grpc gitlab # [ 429.604058] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.448Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"dbe1c9f856e4076f3eeb16bf","created_at":"2026-09-05T12:00:55.118Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-c2ac31ef729f40a2","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.125Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3481,"job_status":"rescheduled","message":"PipelineProcessWorker JID-dbe1c9f856e4076f3eeb16bf: rescheduled"} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.14 seconds) gitlab # [ 429.618496] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.456Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"dbe1c9f856e4076f3eeb16bf","created_at":"2026-09-05T12:00:55.118Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-c2ac31ef729f40a2","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.125Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-dbe1c9f856e4076f3eeb16bf: done: 0.462297 sec","job_status":"done","queue_duration_s":0.869206,"scheduling_latency_s":0.869225,"gitaly_calls":3,"gitaly_duration_s":0.179124,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.021889,"redis_read_bytes":151,"redis_write_bytes":18016,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000835,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001418,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004437,"redis_queues_read_bytes":29,"redis_queues_write_bytes":12315,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.008986,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.0004,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001581,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003397,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.215382,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.406931,"duration_s":0.462297,"completed_at":"2026-09-05T12:00:56.456Z","load_balancing_strategy":"primary","db_duration_s":0.058353,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 429.669067] gitaly[897]: time="2026-09-05T12:00:55.788Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:55.788" grpc.time_ms=0.424 pid=897 span.kind=server system=grpc gitlab # [ 429.678400] gitaly[897]: time="2026-09-05T12:00:55.794Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z diskcache=87f5e8bf-ff7e-4151-a80f-387f109c687e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-05T12:01:05.172" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:55.733" pid=897 ref=refs/keep-around/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 remote_ip="2001:db8:1::1" revision=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 span.kind=server system=grpc username=root gitlab # [ 429.689659] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 98 gitlab # [ 429.692221] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.458Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4cb30c3895b45723f4bb34b1"],"class":"WebHooks::LogExecutionWorker","jid":"8be7c91a60ecca8755f79dd2","created_at":"2026-09-05T12:00:55.213Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-96d04be0e92b41bf","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:18c4d47a372e632803289480424fcc8fbed1790f821f52394b4deebdeed96a3a","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.217Z","job_size_bytes":2591,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8be7c91a60ecca8755f79dd2: start","job_status":"start","queue_duration_s":1.241037,"scheduling_latency_s":1.241064} gitlab # [ 429.709096] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.487Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4cb30c3895b45723f4bb34b1"],"class":"WebHooks::LogExecutionWorker","jid":"8be7c91a60ecca8755f79dd2","created_at":"2026-09-05T12:00:55.213Z","trace_propagation_headers":{"sentry-trace":"dea36b141df34af48d2836cba257aa80-96d04be0e92b41bf","baggage":"sentry-trace_id=dea36b141df34af48d2836cba257aa80,sentry-release=Unknown"},"correlation_id":"01M1RQ01TB7G690X5WZA7AJ5F2","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:18c4d47a372e632803289480424fcc8fbed1790f821f52394b4deebdeed96a3a","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.217Z","job_size_bytes":2591,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8be7c91a60ecca8755f79dd2: done: 0.028522 sec","job_status":"done","queue_duration_s":1.241037,"scheduling_latency_s":1.241064,"redis_calls":7,"redis_duration_s":0.0011200000000000001,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028522,"completed_at":"2026-09-05T12:00:56.487Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.752455] gitaly[897]: time="2026-09-05T12:00:55.795Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=50 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5413 command.oublock=8 command.real_time_ms=56 command.spawn_token_fork_ms=0 command.system_time_ms=32 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-05T12:01:05.172" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-05T12:00:55.733" grpc.time_ms=61.88 pid=897 ref=refs/keep-around/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 remote_ip="2001:db8:1::1" revision=3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 span.kind=server system=grpc username=root gitlab # [ 429.767075] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.488Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"db8d5552f9eff91974eeaecc","created_at":"2026-09-05T12:00:55.799Z","trace_propagation_headers":{"sentry-trace":"85f5fdf4148946d6a21c53ee83d41eef-09b630b428c64464","baggage":"sentry-trace_id=85f5fdf4148946d6a21c53ee83d41eef,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2WSYKBXV8Y7QKG7Q7B8Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.804Z","job_size_bytes":65,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-db8d5552f9eff91974eeaecc: start","job_status":"start","queue_duration_s":0.684177,"scheduling_latency_s":0.684203} gitlab # [ 429.783955] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.525Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"db8d5552f9eff91974eeaecc","created_at":"2026-09-05T12:00:55.799Z","trace_propagation_headers":{"sentry-trace":"85f5fdf4148946d6a21c53ee83d41eef-09b630b428c64464","baggage":"sentry-trace_id=85f5fdf4148946d6a21c53ee83d41eef,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2WSYKBXV8Y7QKG7Q7B8Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.804Z","job_size_bytes":65,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-db8d5552f9eff91974eeaecc: done: 0.036575 sec","job_status":"done","queue_duration_s":0.684177,"scheduling_latency_s":0.684203,"redis_calls":7,"redis_duration_s":0.00109,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036575,"completed_at":"2026-09-05T12:00:56.524Z","load_balancing_strategy":"primary","db_duration_s":0.007592,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 429.826737] gitaly[897]: time="2026-09-05T12:00:55.798Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:25.797" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:55.797" grpc.time_ms=0.295 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.839573] gitaly[897]: time="2026-09-05T12:00:56.160Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:56.159" grpc.time_ms=0.715 pid=897 span.kind=server system=grpc gitlab # [ 429.849211] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.527Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d8ffcbe43ca2a0f461f80ae5","created_at":"2026-09-05T12:00:55.805Z","trace_propagation_headers":{"sentry-trace":"85f5fdf4148946d6a21c53ee83d41eef-09b630b428c64464","baggage":"sentry-trace_id=85f5fdf4148946d6a21c53ee83d41eef,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2WSYKBXV8Y7QKG7Q7B8Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.808Z","job_size_bytes":65,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d8ffcbe43ca2a0f461f80ae5: start","job_status":"start","queue_duration_s":0.719892,"scheduling_latency_s":0.719918} gitlab # [ 429.867798] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.590Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d8ffcbe43ca2a0f461f80ae5","created_at":"2026-09-05T12:00:55.805Z","trace_propagation_headers":{"sentry-trace":"85f5fdf4148946d6a21c53ee83d41eef-09b630b428c64464","baggage":"sentry-trace_id=85f5fdf4148946d6a21c53ee83d41eef,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2WSYKBXV8Y7QKG7Q7B8Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.808Z","job_size_bytes":65,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d8ffcbe43ca2a0f461f80ae5: done: 0.062538 sec","job_status":"done","queue_duration_s":0.719892,"scheduling_latency_s":0.719918,"redis_calls":10,"redis_duration_s":0.00277,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000857,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.062538,"completed_at":"2026-09-05T12:00:56.590Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.911632] gitaly[897]: time="2026-09-05T12:00:56.210Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:56.208" grpc.time_ms=0.771 pid=897 span.kind=server system=grpc gitlab # [ 429.921158] gitaly[897]: time="2026-09-05T12:00:56.263Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-05T12:00:56.262" grpc.time_ms=1.056 pid=897 span.kind=server system=grpc gitlab # [ 429.930773] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.592Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"56f085e6bd890e33d89817ca","created_at":"2026-09-05T12:00:55.866Z","trace_propagation_headers":{"sentry-trace":"85f5fdf4148946d6a21c53ee83d41eef-09b630b428c64464","baggage":"sentry-trace_id=85f5fdf4148946d6a21c53ee83d41eef,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2WSYKBXV8Y7QKG7Q7B8Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.869Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56f085e6bd890e33d89817ca: start","job_status":"start","queue_duration_s":0.722746,"scheduling_latency_s":0.722781} gitlab # [ 429.946754] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.770Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"56f085e6bd890e33d89817ca","created_at":"2026-09-05T12:00:55.866Z","trace_propagation_headers":{"sentry-trace":"85f5fdf4148946d6a21c53ee83d41eef-09b630b428c64464","baggage":"sentry-trace_id=85f5fdf4148946d6a21c53ee83d41eef,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1RQ2WSYKBXV8Y7QKG7Q7B8Z","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.869Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56f085e6bd890e33d89817ca: done: 0.177871 sec","job_status":"done","queue_duration_s":0.722746,"scheduling_latency_s":0.722781,"gitaly_calls":2,"gitaly_duration_s":0.007701,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008565,"redis_read_bytes":80,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001453,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001322,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000363,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001908,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000363,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000579,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.127323,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13061784,"duration_s":0.177871,"completed_at":"2026-09-05T12:00:56.770Z","load_balancing_strategy":"primary","db_duration_s":0.042904,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 429.997935] gitaly[897]: time="2026-09-05T12:00:56.274Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 diskcache=00767066-7c06-4d5f-b51a-f04d806ebea7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-05T12:01:26.107" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:56.107" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.007820] gitaly[897]: time="2026-09-05T12:00:56.275Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=181 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7394 command.oublock=0 command.real_time_ms=161 command.spawn_token_fork_ms=1 command.system_time_ms=108 command.user_time_ms=73 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-05T12:01:26.107" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:00:56.107" grpc.time_ms=167.116 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.020957] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.772Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"6d61eb5d8bf7216e988f87af","created_at":"2026-09-05T12:00:55.900Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-fb2bb1a6ceab41b7","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.907Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-6d61eb5d8bf7216e988f87af: start","job_status":"start","queue_duration_s":0.865137,"scheduling_latency_s":0.865161} gitlab # [ 430.037847] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.885Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"6d61eb5d8bf7216e988f87af","created_at":"2026-09-05T12:00:55.900Z","trace_propagation_headers":{"sentry-trace":"64a1ffbe6c52413f9e4be5f5e895c79a-fb2bb1a6ceab41b7","baggage":"sentry-trace_id=64a1ffbe6c52413f9e4be5f5e895c79a,sentry-release=Unknown"},"correlation_id":"01M1RPZW09FP863AAXMJ1HX43F","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:55.907Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-6d61eb5d8bf7216e988f87af: done: 0.112858 sec","job_status":"done","queue_duration_s":0.865137,"scheduling_latency_s":0.865161,"redis_calls":11,"redis_duration_s":0.0032110000000000003,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000559,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002095,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000557,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037758,"duration_s":0.112858,"completed_at":"2026-09-05T12:00:56.885Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.083403] gitaly[897]: time="2026-09-05T12:00:56.378Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-05T12:01:26.375" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:56.376" grpc.time_ms=2.422 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.096326] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.887Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8cba482136af05f08183203f","created_at":"2026-09-05T12:00:56.094Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.098Z","job_size_bytes":23,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8cba482136af05f08183203f: start","job_status":"start","queue_duration_s":0.788931,"scheduling_latency_s":0.788952} gitlab # [ 430.112730] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.938Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8cba482136af05f08183203f","created_at":"2026-09-05T12:00:56.094Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.098Z","job_size_bytes":23,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8cba482136af05f08183203f: done: 0.05138 sec","job_status":"done","queue_duration_s":0.788931,"scheduling_latency_s":0.788952,"redis_calls":7,"redis_duration_s":0.002399,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05138,"completed_at":"2026-09-05T12:00:56.938Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012626,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 430.154834] gitaly[897]: time="2026-09-05T12:00:56.437Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2VS8H9PC078XVC6687J2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:26.436" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:56.436" grpc.time_ms=0.549 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.167324] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.940Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9e2f2dba6e5860d95c04a983","created_at":"2026-09-05T12:00:56.283Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.287Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9e2f2dba6e5860d95c04a983: start","job_status":"start","queue_duration_s":0.653046,"scheduling_latency_s":0.653068} gitlab # [ 430.184126] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.974Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9e2f2dba6e5860d95c04a983","created_at":"2026-09-05T12:00:56.283Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.287Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9e2f2dba6e5860d95c04a983: done: 0.034111 sec","job_status":"done","queue_duration_s":0.653046,"scheduling_latency_s":0.653068,"redis_calls":7,"redis_duration_s":0.001973,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001412,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000561,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034111,"completed_at":"2026-09-05T12:00:56.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.226242] gitaly[897]: time="2026-09-05T12:00:56.493Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2XKB6M6XR118109KRFWG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:26.492" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:56.492" grpc.time_ms=0.411 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.239019] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:56.976Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5e72dfa14b142732dac5bdd4","created_at":"2026-09-05T12:00:56.298Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.303Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5e72dfa14b142732dac5bdd4: start","job_status":"start","queue_duration_s":0.672421,"scheduling_latency_s":0.672441} gitlab # [ 430.254744] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.011Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5e72dfa14b142732dac5bdd4","created_at":"2026-09-05T12:00:56.298Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.303Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5e72dfa14b142732dac5bdd4: done: 0.035003 sec","job_status":"done","queue_duration_s":0.672421,"scheduling_latency_s":0.672441,"redis_calls":7,"redis_duration_s":0.002009,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035003,"completed_at":"2026-09-05T12:00:57.011Z","load_balancing_strategy":"primary","db_duration_s":0.008531,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.298223] gitaly[897]: time="2026-09-05T12:00:56.701Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-05T12:01:26.699" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:56.700" grpc.time_ms=1.17 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.310757] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.013Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"6cfefb9fcfd329560f3dd0b5","created_at":"2026-09-05T12:00:56.328Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.332Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-6cfefb9fcfd329560f3dd0b5: start","job_status":"start","queue_duration_s":0.680386,"scheduling_latency_s":0.680406} gitlab # [ 430.326337] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.049Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"6cfefb9fcfd329560f3dd0b5","created_at":"2026-09-05T12:00:56.328Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.332Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-6cfefb9fcfd329560f3dd0b5: done: 0.036139 sec","job_status":"done","queue_duration_s":0.680386,"scheduling_latency_s":0.680406,"redis_calls":10,"redis_duration_s":0.002742,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001351,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.0008,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036139,"completed_at":"2026-09-05T12:00:57.049Z","load_balancing_strategy":"primary","db_duration_s":0.00835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.368763] gitaly[897]: time="2026-09-05T12:00:56.756Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2WSYKBXV8Y7QKG7Q7B8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:26.754" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:56.754" grpc.time_ms=0.975 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.381122] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.050Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a86eb76c073ea8cceecc149a","created_at":"2026-09-05T12:00:56.333Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.337Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a86eb76c073ea8cceecc149a: start","job_status":"start","queue_duration_s":0.712956,"scheduling_latency_s":0.712974} gitlab # [ 430.397384] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.080Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a86eb76c073ea8cceecc149a","created_at":"2026-09-05T12:00:56.333Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.337Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a86eb76c073ea8cceecc149a: done: 0.02987 sec","job_status":"done","queue_duration_s":0.712956,"scheduling_latency_s":0.712974,"redis_calls":7,"redis_duration_s":0.001976,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000516,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02987,"completed_at":"2026-09-05T12:00:57.080Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.439340] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1f140161ddcd34333dc23132","created_at":"2026-09-05T12:00:56.338Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.343Z","job_size_bytes":19,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1f140161ddcd34333dc23132: start","job_status":"start","queue_duration_s":0.738188,"scheduling_latency_s":0.738207} gitlab # [ 430.455528] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.109Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1f140161ddcd34333dc23132","created_at":"2026-09-05T12:00:56.338Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.343Z","job_size_bytes":19,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1f140161ddcd34333dc23132: done: 0.027893 sec","job_status":"done","queue_duration_s":0.738188,"scheduling_latency_s":0.738207,"redis_calls":8,"redis_duration_s":0.002138,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001627,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000511,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027893,"completed_at":"2026-09-05T12:00:57.109Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.497642] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.111Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e69e3309f50ead13182211d7","created_at":"2026-09-05T12:00:56.345Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.350Z","job_size_bytes":66,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e69e3309f50ead13182211d7: start","job_status":"start","queue_duration_s":0.761091,"scheduling_latency_s":0.761111} gitlab # [ 430.514271] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.136Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e69e3309f50ead13182211d7","created_at":"2026-09-05T12:00:56.345Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.350Z","job_size_bytes":66,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e69e3309f50ead13182211d7: done: 0.024884 sec","job_status":"done","queue_duration_s":0.761091,"scheduling_latency_s":0.761111,"redis_calls":7,"redis_duration_s":0.0018150000000000002,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000514,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024884,"completed_at":"2026-09-05T12:00:57.136Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.556145] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.137Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"28431343f45e38b9d43e975f","created_at":"2026-09-05T12:00:56.351Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.354Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-28431343f45e38b9d43e975f: start","job_status":"start","queue_duration_s":0.783749,"scheduling_latency_s":0.783767} gitlab # [ 430.571677] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.167Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"28431343f45e38b9d43e975f","created_at":"2026-09-05T12:00:56.351Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.354Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-28431343f45e38b9d43e975f: done: 0.02944 sec","job_status":"done","queue_duration_s":0.783749,"scheduling_latency_s":0.783767,"redis_calls":7,"redis_duration_s":0.001957,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001436,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02944,"completed_at":"2026-09-05T12:00:57.167Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007736,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.613046] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.169Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d2bac0fa5454b86d980563f4","created_at":"2026-09-05T12:00:56.354Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.357Z","job_size_bytes":6,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d2bac0fa5454b86d980563f4: start","job_status":"start","queue_duration_s":0.811802,"scheduling_latency_s":0.811822} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 430.628222] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.183Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d2bac0fa5454b86d980563f4","created_at":"2026-09-05T12:00:56.354Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.357Z","job_size_bytes":6,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d2bac0fa5454b86d980563f4: done: 0.014976 sec","job_status":"done","queue_duration_s":0.811802,"scheduling_latency_s":0.811822,"redis_calls":7,"redis_duration_s":0.00165,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014976,"completed_at":"2026-09-05T12:00:57.183Z","load_balancing_strategy":"primary","db_duration_s":0.00065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.668548] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c5206f2e78b24fad0099084d","created_at":"2026-09-05T12:00:56.357Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.360Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c5206f2e78b24fad0099084d: start","job_status":"start","queue_duration_s":0.825137,"scheduling_latency_s":0.825154} gitlab # [ 430.682478] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.212Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c5206f2e78b24fad0099084d","created_at":"2026-09-05T12:00:56.357Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.360Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c5206f2e78b24fad0099084d: done: 0.026749 sec","job_status":"done","queue_duration_s":0.825137,"scheduling_latency_s":0.825154,"redis_calls":7,"redis_duration_s":0.001862,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026749,"completed_at":"2026-09-05T12:00:57.212Z","load_balancing_strategy":"primary","db_duration_s":0.005819,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.723508] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.213Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"bcee6d59b47a4e02de938d00","created_at":"2026-09-05T12:00:56.449Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.450Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bcee6d59b47a4e02de938d00: start","job_status":"start","queue_duration_s":0.762614,"scheduling_latency_s":0.762633} gitlab # [ 430.738942] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.242Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"bcee6d59b47a4e02de938d00","created_at":"2026-09-05T12:00:56.449Z","trace_propagation_headers":{"sentry-trace":"26f4e6887cea46b9a9b6b6b784358164-8163efc85f254fc5","baggage":"sentry-trace_id=26f4e6887cea46b9a9b6b6b784358164,sentry-release=Unknown"},"correlation_id":"01M1RQ2VS8H9PC078XVC6687J2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.450Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bcee6d59b47a4e02de938d00: done: 0.02925 sec","job_status":"done","queue_duration_s":0.762614,"scheduling_latency_s":0.762633,"redis_calls":8,"redis_duration_s":0.00211,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001641,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02925,"completed_at":"2026-09-05T12:00:57.242Z","load_balancing_strategy":"primary","db_duration_s":0.006255,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 430.779970] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.244Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"010b0b45526f2e2b7bb83010","created_at":"2026-09-05T12:00:56.678Z","trace_propagation_headers":{"sentry-trace":"85f5fdf4148946d6a21c53ee83d41eef-fa27bf925bb443f1","baggage":"sentry-trace_id=85f5fdf4148946d6a21c53ee83d41eef,sentry-release=Unknown"},"correlation_id":"01M1RQ2WSYKBXV8Y7QKG7Q7B8Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.681Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-010b0b45526f2e2b7bb83010: start","job_status":"start","queue_duration_s":0.56278,"scheduling_latency_s":0.562797} gitlab # [ 430.796150] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:57.311Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"010b0b45526f2e2b7bb83010","created_at":"2026-09-05T12:00:56.678Z","trace_propagation_headers":{"sentry-trace":"85f5fdf4148946d6a21c53ee83d41eef-fa27bf925bb443f1","baggage":"sentry-trace_id=85f5fdf4148946d6a21c53ee83d41eef,sentry-release=Unknown"},"correlation_id":"01M1RQ2WSYKBXV8Y7QKG7Q7B8Z","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:56.681Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-010b0b45526f2e2b7bb83010: done: 0.067435 sec","job_status":"done","queue_duration_s":0.56278,"scheduling_latency_s":0.562797,"redis_calls":7,"redis_duration_s":0.0019140000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067435,"completed_at":"2026-09-05T12:00:57.311Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.849608] gitaly[897]: time="2026-09-05T12:00:57.732Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2YRRDBXG60HG0VQVK78D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:27.130" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:57.731" grpc.time_ms=0.994 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.913710] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 171 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.27 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 431.963034] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:58.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"406f6a57d7c998263f720a81","created_at":"2026-09-05T12:00:58.812Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2932fc025ddac47f3062b0671556f744","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:58.840Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-406f6a57d7c998263f720a81: start","job_status":"start","queue_duration_s":0.003889,"scheduling_latency_s":0.003948} gitlab # [ 432.087513] gitaly[897]: time="2026-09-05T12:00:58.968Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ2ZZBD09SJE88Q0WNR9Q2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:28.066" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:00:58.966" grpc.time_ms=1.028 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.22 seconds) gitlab # [ 432.126733] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"406f6a57d7c998263f720a81","created_at":"2026-09-05T12:00:58.812Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2932fc025ddac47f3062b0671556f744","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:58.840Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-406f6a57d7c998263f720a81: done: 0.162274 sec","job_status":"done","queue_duration_s":0.003889,"scheduling_latency_s":0.003948,"redis_calls":11,"redis_duration_s":0.02385,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.008924,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00988,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005046,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.162274,"completed_at":"2026-09-05T12:00:59.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006428,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.203528] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"b5ed511be9fcfcb43f59a0d5","created_at":"2026-09-05T12:00:58.865Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"daa0ed8a2b9b2ecb3f6a6404e1ba479e","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:58.880Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-b5ed511be9fcfcb43f59a0d5: start","job_status":"start","queue_duration_s":0.130055,"scheduling_latency_s":0.130103} gitlab # [ 432.219722] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:00:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 151 gitlab # [ 432.222721] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"b5ed511be9fcfcb43f59a0d5","created_at":"2026-09-05T12:00:58.865Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"daa0ed8a2b9b2ecb3f6a6404e1ba479e","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:58.880Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-b5ed511be9fcfcb43f59a0d5: done: 0.094211 sec","job_status":"done","queue_duration_s":0.130055,"scheduling_latency_s":0.130103,"redis_calls":10,"redis_duration_s":0.008026,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001822,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001356,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0015662,"duration_s":0.094211,"completed_at":"2026-09-05T12:00:59.104Z","load_balancing_strategy":"primary","db_duration_s":0.014527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.263907] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ac9427c206af43b39e2bef6b","created_at":"2026-09-05T12:00:58.893Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54b8b2b67134f456a418f93851d3a9e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:58.912Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ac9427c206af43b39e2bef6b: start","job_status":"start","queue_duration_s":0.203039,"scheduling_latency_s":0.203087} gitlab # [ 432.295265] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.178Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 432.344598] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.226Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ac9427c206af43b39e2bef6b","created_at":"2026-09-05T12:00:58.893Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54b8b2b67134f456a418f93851d3a9e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:58.912Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ac9427c206af43b39e2bef6b: done: 0.111757 sec","job_status":"done","queue_duration_s":0.203039,"scheduling_latency_s":0.203087,"redis_calls":9,"redis_duration_s":0.021158999999999997,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005927,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010162,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00507,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.111757,"completed_at":"2026-09-05T12:00:59.226Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002281,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.425598] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.236Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a33f0a14fda13e21c96182c0","created_at":"2026-09-05T12:00:58.934Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8ce8892a067b79cbac2c87631ff7403","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:58.962Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a33f0a14fda13e21c96182c0: start","job_status":"start","queue_duration_s":0.274847,"scheduling_latency_s":0.274896} gitlab # [ 432.442441] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.320Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 432.472551] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a33f0a14fda13e21c96182c0","created_at":"2026-09-05T12:00:58.934Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8ce8892a067b79cbac2c87631ff7403","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:58.962Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a33f0a14fda13e21c96182c0: done: 0.11804 sec","job_status":"done","queue_duration_s":0.274847,"scheduling_latency_s":0.274896,"redis_calls":10,"redis_duration_s":0.018945,"redis_read_bytes":41,"redis_write_bytes":1140,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.004194,"redis_feature_flag_read_bytes":36,"redis_feature_flag_write_bytes":374,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003399,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.11804,"completed_at":"2026-09-05T12:00:59.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.553129] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"815eded27d9c4cfd83fd00a3","created_at":"2026-09-05T12:00:58.993Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aaef3809f0a038c2eb4b8bfd1a1d6485","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.068Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-815eded27d9c4cfd83fd00a3: start","job_status":"start","queue_duration_s":0.304115,"scheduling_latency_s":0.304166} gitlab # [ 432.599587] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.482Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"815eded27d9c4cfd83fd00a3","created_at":"2026-09-05T12:00:58.993Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aaef3809f0a038c2eb4b8bfd1a1d6485","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.068Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-815eded27d9c4cfd83fd00a3: done: 0.109127 sec","job_status":"done","queue_duration_s":0.304115,"scheduling_latency_s":0.304166,"redis_calls":11,"redis_duration_s":0.02563,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007818,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010995,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006817,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0019151,"duration_s":0.109127,"completed_at":"2026-09-05T12:00:59.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.687459] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.489Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"54f04ae283c7d110ddea6af4","created_at":"2026-09-05T12:00:59.092Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"a72b0d6b9b89cbff65f1bbc649bef385","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.108Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-54f04ae283c7d110ddea6af4: start","job_status":"start","queue_duration_s":0.38122,"scheduling_latency_s":0.381285} gitlab # [ 432.728643] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.574Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"54f04ae283c7d110ddea6af4","created_at":"2026-09-05T12:00:59.092Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"a72b0d6b9b89cbff65f1bbc649bef385","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.108Z","job_size_bytes":3,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-54f04ae283c7d110ddea6af4: done: 0.085299 sec","job_status":"done","queue_duration_s":0.38122,"scheduling_latency_s":0.381285,"redis_calls":10,"redis_duration_s":0.013254,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0039,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005897,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00141366,"duration_s":0.085299,"completed_at":"2026-09-05T12:00:59.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.817481] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"fb49fab1f5046776414d4368","created_at":"2026-09-05T12:00:59.122Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe2dc24012eb93af260f99d652c1283b","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.136Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-fb49fab1f5046776414d4368: start","job_status":"start","queue_duration_s":0.445749,"scheduling_latency_s":0.445802} gitlab # [ 432.837842] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"fb49fab1f5046776414d4368","created_at":"2026-09-05T12:00:59.122Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe2dc24012eb93af260f99d652c1283b","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.136Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-fb49fab1f5046776414d4368: done: 0.138035 sec","job_status":"done","queue_duration_s":0.445749,"scheduling_latency_s":0.445802,"redis_calls":13,"redis_duration_s":0.032372,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004724,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012839,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.014809,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00172308,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.138035,"completed_at":"2026-09-05T12:00:59.720Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.058613] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:00:59.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c214ea455b8f24f95549cd4a","created_at":"2026-09-05T12:00:59.160Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb3bd5c9e13522701f7011dae93dde30","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.188Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c214ea455b8f24f95549cd4a: start","job_status":"start","queue_duration_s":0.751361,"scheduling_latency_s":0.751389} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 433.353675] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c214ea455b8f24f95549cd4a","created_at":"2026-09-05T12:00:59.160Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb3bd5c9e13522701f7011dae93dde30","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.188Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c214ea455b8f24f95549cd4a: done: 0.29491 sec","job_status":"done","queue_duration_s":0.751361,"scheduling_latency_s":0.751389,"redis_calls":11,"redis_duration_s":0.028376,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005454,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.015629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007293,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00489836,"exclusive_lock_hold_duration_s":0.14945084,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.29491,"completed_at":"2026-09-05T12:01:00.235Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.127934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.445097] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"197a7b9b7fbb0ad7d96b8f06","created_at":"2026-09-05T12:00:59.216Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0ef9191ef9e5886b79454a7ca4a9aa8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:38f97efb1ac6a87ded5913833f7d951478e91a6be2a4328abbe154ee188cf7fb","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.234Z","job_size_bytes":101,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-197a7b9b7fbb0ad7d96b8f06: start","job_status":"start","queue_duration_s":1.011057,"scheduling_latency_s":1.011103} gitlab # [ 433.462466] gitaly[897]: time="2026-09-05T12:01:00.335Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ315SA9R0WQDD5S278PPT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:30.033" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:01:00.334" grpc.time_ms=0.581 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.496525] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 289 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.36 seconds) gitlab # [ 433.582750] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.463Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"197a7b9b7fbb0ad7d96b8f06","created_at":"2026-09-05T12:00:59.216Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0ef9191ef9e5886b79454a7ca4a9aa8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:38f97efb1ac6a87ded5913833f7d951478e91a6be2a4328abbe154ee188cf7fb","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.234Z","job_size_bytes":101,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-197a7b9b7fbb0ad7d96b8f06: done: 0.218561 sec","job_status":"done","queue_duration_s":1.011057,"scheduling_latency_s":1.011103,"redis_calls":10,"redis_duration_s":0.011806,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00086,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004571,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.058,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0017518,"duration_s":0.218561,"completed_at":"2026-09-05T12:01:00.463Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.653105] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.474Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"37f3d78e63c17216c2e3c006","created_at":"2026-09-05T12:00:59.244Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d6fb984797f38ff2b2b049e12f7a838","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.257Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-37f3d78e63c17216c2e3c006: start","job_status":"start","queue_duration_s":1.21686,"scheduling_latency_s":1.216912} gitlab # [ 433.669970] nix-daemon[3765]: accepted connection from pid 4510, user nixbot gitlab # [ 433.686726] nix-daemon[3765]: accepted connection from pid 4472, user nixbot gitlab # [ 433.736825] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.619Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"37f3d78e63c17216c2e3c006","created_at":"2026-09-05T12:00:59.244Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d6fb984797f38ff2b2b049e12f7a838","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.257Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-37f3d78e63c17216c2e3c006: done: 0.145316 sec","job_status":"done","queue_duration_s":1.21686,"scheduling_latency_s":1.216912,"redis_calls":10,"redis_duration_s":0.006149,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000518,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004356,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001275,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0011279,"duration_s":0.145316,"completed_at":"2026-09-05T12:01:00.619Z","load_balancing_strategy":"primary","db_duration_s":0.00522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.816283] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"54dacc4af58038d77b2a4d6a","created_at":"2026-09-05T12:00:59.264Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5f1869ed1abe25aee28ff68141fe8a8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.288Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-54dacc4af58038d77b2a4d6a: start","job_status":"start","queue_duration_s":1.333685,"scheduling_latency_s":1.33372} gitlab # [ 433.839306] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"54dacc4af58038d77b2a4d6a","created_at":"2026-09-05T12:00:59.264Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5f1869ed1abe25aee28ff68141fe8a8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.288Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-54dacc4af58038d77b2a4d6a: done: 0.099156 sec","job_status":"done","queue_duration_s":1.333685,"scheduling_latency_s":1.33372,"redis_calls":9,"redis_duration_s":0.0239,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003782,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.018641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001477,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.099156,"completed_at":"2026-09-05T12:01:00.720Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011794,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.885107] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.731Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"a8b59c5901bdda79c60c7e75","created_at":"2026-09-05T12:00:59.317Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0a8cbbe817c994c394a78aea900f19e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.349Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-a8b59c5901bdda79c60c7e75: start","job_status":"start","queue_duration_s":1.382427,"scheduling_latency_s":1.382481} gitlab # [ 433.981514] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"a8b59c5901bdda79c60c7e75","created_at":"2026-09-05T12:00:59.317Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0a8cbbe817c994c394a78aea900f19e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.349Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-a8b59c5901bdda79c60c7e75: done: 0.132404 sec","job_status":"done","queue_duration_s":1.382427,"scheduling_latency_s":1.382481,"redis_calls":10,"redis_duration_s":0.025613999999999998,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004669,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016522,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00219906,"duration_s":0.132404,"completed_at":"2026-09-05T12:01:00.863Z","load_balancing_strategy":"primary","db_duration_s":0.002739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.070441] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:00.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"3a48b78512762efb433cf7d8","created_at":"2026-09-05T12:00:59.372Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b858b094538a76d263bcdcc2fb3cbfa","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.389Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-3a48b78512762efb433cf7d8: start","job_status":"start","queue_duration_s":1.49507,"scheduling_latency_s":1.495119} gitlab # [ 434.192138] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"3a48b78512762efb433cf7d8","created_at":"2026-09-05T12:00:59.372Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b858b094538a76d263bcdcc2fb3cbfa","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.389Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-3a48b78512762efb433cf7d8: done: 0.190048 sec","job_status":"done","queue_duration_s":1.49507,"scheduling_latency_s":1.495119,"redis_calls":10,"redis_duration_s":0.018728,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002703,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003654,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00177328,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00799824},"duration_s":0.190048,"completed_at":"2026-09-05T12:01:01.074Z","load_balancing_strategy":"primary","db_duration_s":0.044245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.284624] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.084Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"6ff3ea9aac8ff9668cee7979","created_at":"2026-09-05T12:00:59.396Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0be40828f2bbe6debe26085cc271bd16","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.419Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-6ff3ea9aac8ff9668cee7979: start","job_status":"start","queue_duration_s":1.665253,"scheduling_latency_s":1.665297} gitlab # [ 434.351506] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.234Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"6ff3ea9aac8ff9668cee7979","created_at":"2026-09-05T12:00:59.396Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0be40828f2bbe6debe26085cc271bd16","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.419Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-6ff3ea9aac8ff9668cee7979: done: 0.149797 sec","job_status":"done","queue_duration_s":1.665253,"scheduling_latency_s":1.665297,"redis_calls":20,"redis_duration_s":0.037297,"redis_read_bytes":23,"redis_write_bytes":4317,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.024522,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.001335,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2022,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.008555,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002885,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00059122,"duration_s":0.149797,"completed_at":"2026-09-05T12:01:01.233Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004682,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.446222] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.236Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1dfba9b9f5d112e45f0c88c4","created_at":"2026-09-05T12:00:59.446Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fee2e75399b73bf23267a74389ba0b4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.463Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1dfba9b9f5d112e45f0c88c4: start","job_status":"start","queue_duration_s":1.773581,"scheduling_latency_s":1.773614} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 434.656257] gitaly[897]: time="2026-09-05T12:01:01.538Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ32FZ4MH1FR97E5K8SM1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:31.536" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:01:01.537" grpc.time_ms=0.938 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.21 seconds) gitlab # [ 434.709078] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 152 gitlab # [ 434.770460] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.653Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1dfba9b9f5d112e45f0c88c4","created_at":"2026-09-05T12:00:59.446Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fee2e75399b73bf23267a74389ba0b4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.463Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1dfba9b9f5d112e45f0c88c4: done: 0.41687 sec","job_status":"done","queue_duration_s":1.773581,"scheduling_latency_s":1.773614,"redis_calls":564,"redis_duration_s":0.09430899999999999,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000578,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.092329,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.358369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.41687,"completed_at":"2026-09-05T12:01:01.653Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001022,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.869224] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.655Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"43ab5bd6573afa2bb5ca43f2","created_at":"2026-09-05T12:00:59.483Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"846d56f8c9b91373435d59e9fde0424c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.494Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-43ab5bd6573afa2bb5ca43f2: start","job_status":"start","queue_duration_s":2.160862,"scheduling_latency_s":2.160886} gitlab # [ 434.891031] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.679Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"43ab5bd6573afa2bb5ca43f2","created_at":"2026-09-05T12:00:59.483Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"846d56f8c9b91373435d59e9fde0424c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.494Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-43ab5bd6573afa2bb5ca43f2: done: 0.024267 sec","job_status":"done","queue_duration_s":2.160862,"scheduling_latency_s":2.160886,"redis_calls":11,"redis_duration_s":0.0025729999999999998,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053494,"exclusive_lock_hold_duration_s":0.00359482,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.024267,"completed_at":"2026-09-05T12:01:01.679Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.938707] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.680Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f045b5105d2ca49d28b0af0f","created_at":"2026-09-05T12:00:59.505Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44875b6d95e80a304ab9148a6dda8516","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.514Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f045b5105d2ca49d28b0af0f: start","job_status":"start","queue_duration_s":2.165916,"scheduling_latency_s":2.165934} gitlab # [ 434.952406] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.710Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f045b5105d2ca49d28b0af0f","created_at":"2026-09-05T12:00:59.505Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44875b6d95e80a304ab9148a6dda8516","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.514Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f045b5105d2ca49d28b0af0f: done: 0.029619 sec","job_status":"done","queue_duration_s":2.165916,"scheduling_latency_s":2.165934,"redis_calls":9,"redis_duration_s":0.00241,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000533,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029619,"completed_at":"2026-09-05T12:01:01.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006955,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.993287] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"89db1e7ff5c7860f3c82e0dd","created_at":"2026-09-05T12:00:59.538Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97e837eccb4fd6de9f169a20d84638aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.557Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-89db1e7ff5c7860f3c82e0dd: start","job_status":"start","queue_duration_s":2.153967,"scheduling_latency_s":2.153985} gitlab # [ 435.006723] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.741Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"89db1e7ff5c7860f3c82e0dd","created_at":"2026-09-05T12:00:59.538Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97e837eccb4fd6de9f169a20d84638aa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.557Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-89db1e7ff5c7860f3c82e0dd: done: 0.029506 sec","job_status":"done","queue_duration_s":2.153967,"scheduling_latency_s":2.153985,"redis_calls":11,"redis_duration_s":0.0033439999999999998,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001583,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00123,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000577,"exclusive_lock_hold_duration_s":0.00837696,"duration_s":0.029506,"completed_at":"2026-09-05T12:01:01.741Z","load_balancing_strategy":"primary","db_duration_s":0.00536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.047142] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0b7e47b50e46b847457d5e33","created_at":"2026-09-05T12:00:59.576Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"783e7458e573951a414d4ac4718f35f8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.593Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0b7e47b50e46b847457d5e33: start","job_status":"start","queue_duration_s":2.149555,"scheduling_latency_s":2.149573} gitlab # [ 435.062545] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0b7e47b50e46b847457d5e33","created_at":"2026-09-05T12:00:59.576Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"783e7458e573951a414d4ac4718f35f8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.593Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0b7e47b50e46b847457d5e33: done: 0.033163 sec","job_status":"done","queue_duration_s":2.149555,"scheduling_latency_s":2.149573,"redis_calls":12,"redis_duration_s":0.0032329999999999998,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001577,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001135,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01041268,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.033163,"completed_at":"2026-09-05T12:01:01.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004353,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 435.106494] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PruneOldEventsWorker","jid":"15f5813d1426c58ecd116043","created_at":"2026-09-05T12:00:59.616Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1458aa59d72018ce36fe7591cf489c95","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:aa7847a0e96d8dccea25d7c51fc9fba0c73a3fc691400b9310229c28e3c4fab3","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.641Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PruneOldEventsWorker JID-15f5813d1426c58ecd116043: start","job_status":"start","queue_duration_s":2.136621,"scheduling_latency_s":2.136637} gitlab # [ 435.120283] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PruneOldEventsWorker","jid":"15f5813d1426c58ecd116043","created_at":"2026-09-05T12:00:59.616Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1458aa59d72018ce36fe7591cf489c95","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:aa7847a0e96d8dccea25d7c51fc9fba0c73a3fc691400b9310229c28e3c4fab3","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.641Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PruneOldEventsWorker JID-15f5813d1426c58ecd116043: done: 0.024066 sec","job_status":"done","queue_duration_s":2.136621,"scheduling_latency_s":2.136637,"redis_calls":10,"redis_duration_s":0.002422,"redis_read_bytes":9,"redis_write_bytes":987,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000502,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000476,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003186,"duration_s":0.024066,"completed_at":"2026-09-05T12:01:01.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.161049] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.803Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BulkImports::StaleImportWorker","jid":"a01a92bec44d4187c1d50edb","created_at":"2026-09-05T12:00:59.682Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"956ea6115f0c4282bd9ae67c65cbda81","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1a00ac23ed20191d6dc4c4907281ed3c2c9a9fd8b3894e6dbd87102304111dab","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.703Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"BulkImports::StaleImportWorker JID-a01a92bec44d4187c1d50edb: start","job_status":"start","queue_duration_s":2.099849,"scheduling_latency_s":2.099864} gitlab # [ 435.174549] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.843Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BulkImports::StaleImportWorker","jid":"a01a92bec44d4187c1d50edb","created_at":"2026-09-05T12:00:59.682Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"956ea6115f0c4282bd9ae67c65cbda81","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1a00ac23ed20191d6dc4c4907281ed3c2c9a9fd8b3894e6dbd87102304111dab","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.703Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"BulkImports::StaleImportWorker JID-a01a92bec44d4187c1d50edb: done: 0.040505 sec","job_status":"done","queue_duration_s":2.099849,"scheduling_latency_s":2.099864,"redis_calls":10,"redis_duration_s":0.0029590000000000003,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001858,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043116,"duration_s":0.040505,"completed_at":"2026-09-05T12:01:01.843Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.215809] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.845Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3793edef4f6a340b10f3009f","created_at":"2026-09-05T12:00:59.758Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9251e02b93ec2cc1992544e31079e9f4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.952Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3793edef4f6a340b10f3009f: start","job_status":"start","queue_duration_s":1.892455,"scheduling_latency_s":1.892471} gitlab # [ 435.230214] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.856Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 435.233359] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3793edef4f6a340b10f3009f","created_at":"2026-09-05T12:00:59.758Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9251e02b93ec2cc1992544e31079e9f4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.952Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3793edef4f6a340b10f3009f: done: 0.018375 sec","job_status":"done","queue_duration_s":1.892455,"scheduling_latency_s":1.892471,"redis_calls":9,"redis_duration_s":0.002229,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018375,"completed_at":"2026-09-05T12:01:01.863Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000668,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.275727] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"7d35743965d4f2d85710125f","created_at":"2026-09-05T12:00:59.976Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e24bb542589a18c56b4829b90708e16f","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.999Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-7d35743965d4f2d85710125f: start","job_status":"start","queue_duration_s":1.865196,"scheduling_latency_s":1.865212} gitlab # [ 435.288760] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"7d35743965d4f2d85710125f","created_at":"2026-09-05T12:00:59.976Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e24bb542589a18c56b4829b90708e16f","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-05T12:00:59.999Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-7d35743965d4f2d85710125f: done: 0.031401 sec","job_status":"done","queue_duration_s":1.865196,"scheduling_latency_s":1.865212,"redis_calls":10,"redis_duration_s":0.0026479999999999997,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001666,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000543,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041396,"duration_s":0.031401,"completed_at":"2026-09-05T12:01:01.896Z","load_balancing_strategy":"primary","db_duration_s":0.007499,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.329067] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.897Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::ManageEvidenceWorker","jid":"00469c10c4876950c3dd21d7","created_at":"2026-09-05T12:01:00.026Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14c5bc7c54e1c4c6dd4b7367ddf53cc4","meta.root_caller_id":"Cronjob","meta.feature_category":"release_evidence","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca33015ef91ab20408351751cf9ad5e1289a7d78257a0117231d08455e951d7f","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.045Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::ManageEvidenceWorker JID-00469c10c4876950c3dd21d7: start","job_status":"start","queue_duration_s":1.852863,"scheduling_latency_s":1.852878} gitlab # [ 435.342876] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::ManageEvidenceWorker","jid":"00469c10c4876950c3dd21d7","created_at":"2026-09-05T12:01:00.026Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14c5bc7c54e1c4c6dd4b7367ddf53cc4","meta.root_caller_id":"Cronjob","meta.feature_category":"release_evidence","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca33015ef91ab20408351751cf9ad5e1289a7d78257a0117231d08455e951d7f","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.045Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::ManageEvidenceWorker JID-00469c10c4876950c3dd21d7: done: 0.032937 sec","job_status":"done","queue_duration_s":1.852863,"scheduling_latency_s":1.852878,"redis_calls":10,"redis_duration_s":0.002286,"redis_read_bytes":9,"redis_write_bytes":1055,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000452,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":206,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031198,"duration_s":0.032937,"completed_at":"2026-09-05T12:01:01.930Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.384179] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"8cd630b667298654b9d6ef79","created_at":"2026-09-05T12:01:00.146Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.159Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-8cd630b667298654b9d6ef79: start","job_status":"start","queue_duration_s":1.772999,"scheduling_latency_s":1.773015} gitlab # [ 435.398258] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"8cd630b667298654b9d6ef79","created_at":"2026-09-05T12:01:00.146Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.159Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-8cd630b667298654b9d6ef79: done: 0.036373 sec","job_status":"done","queue_duration_s":1.772999,"scheduling_latency_s":1.773015,"redis_calls":24,"redis_duration_s":0.0058909999999999995,"redis_read_bytes":25,"redis_write_bytes":5972,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001028,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":704,"redis_queues_calls":2,"redis_queues_duration_s":0.000324,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3805,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.00259,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.001949,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032476,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.036373,"completed_at":"2026-09-05T12:01:01.968Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.444954] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.970Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"71889d16c181d15536b56172","created_at":"2026-09-05T12:01:00.172Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efe72fffa2597841e5f51ed012513e17","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.191Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-71889d16c181d15536b56172: start","job_status":"start","queue_duration_s":1.778198,"scheduling_latency_s":1.778214} gitlab # [ 435.458201] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:01.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"71889d16c181d15536b56172","created_at":"2026-09-05T12:01:00.172Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efe72fffa2597841e5f51ed012513e17","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.191Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-71889d16c181d15536b56172: done: 0.028744 sec","job_status":"done","queue_duration_s":1.778198,"scheduling_latency_s":1.778214,"redis_calls":12,"redis_duration_s":0.002899,"redis_read_bytes":11,"redis_write_bytes":1481,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001351,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":548,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052014,"exclusive_lock_hold_duration_s":0.010126099999999999,"extra.ci_pipeline_artifacts_expire_artifacts_worker.destroyed_pipeline_artifacts_count":0,"duration_s":0.028744,"completed_at":"2026-09-05T12:01:01.998Z","load_balancing_strategy":"primary","db_duration_s":0.006211,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.500861] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"e6c1c8a69cb330810e5f2322","created_at":"2026-09-05T12:01:00.212Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.243Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-e6c1c8a69cb330810e5f2322: start","job_status":"start","queue_duration_s":1.756894,"scheduling_latency_s":1.756911} gitlab # [ 435.515133] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"e6c1c8a69cb330810e5f2322","created_at":"2026-09-05T12:01:00.212Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.243Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-e6c1c8a69cb330810e5f2322: done: 0.045024 sec","job_status":"done","queue_duration_s":1.756894,"scheduling_latency_s":1.756911,"redis_calls":28,"redis_duration_s":0.007036,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000895,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000418,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.004337,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001386,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031416,"duration_s":0.045024,"completed_at":"2026-09-05T12:01:02.045Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.558032] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.046Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"8f1619acd19a3b75165350f8","created_at":"2026-09-05T12:01:00.252Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edcc8b6d95aecbdbe7389a887edbc55e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.302Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-8f1619acd19a3b75165350f8: start","job_status":"start","queue_duration_s":1.744538,"scheduling_latency_s":1.744554} gitlab # [ 435.570811] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.066Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"8f1619acd19a3b75165350f8","created_at":"2026-09-05T12:01:00.252Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edcc8b6d95aecbdbe7389a887edbc55e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.302Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-8f1619acd19a3b75165350f8: done: 0.01924 sec","job_status":"done","queue_duration_s":1.744538,"scheduling_latency_s":1.744554,"redis_calls":11,"redis_duration_s":0.002588,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000518,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001315,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030786,"duration_s":0.01924,"completed_at":"2026-09-05T12:01:02.066Z","load_balancing_strategy":"primary","db_duration_s":0.000713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.610724] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.067Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8c63fa6acced6cb94f241422","created_at":"2026-09-05T12:01:00.313Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4122d1106e9e10ac124dd85ff9f32e20","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.326Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8c63fa6acced6cb94f241422: start","job_status":"start","queue_duration_s":1.741133,"scheduling_latency_s":1.741148} gitlab # [ 435.625025] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.108Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8c63fa6acced6cb94f241422","created_at":"2026-09-05T12:01:00.313Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4122d1106e9e10ac124dd85ff9f32e20","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.326Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8c63fa6acced6cb94f241422: done: 0.040581 sec","job_status":"done","queue_duration_s":1.741133,"scheduling_latency_s":1.741148,"redis_calls":9,"redis_duration_s":0.002461,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.040581,"completed_at":"2026-09-05T12:01:02.108Z","load_balancing_strategy":"primary","db_duration_s":0.019361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.667704] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.109Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryArchiveCacheWorker","jid":"eda4699bd12b0fdccf4a780d","created_at":"2026-09-05T12:01:00.344Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d786b6ca12f5d4297c04819dbf5ae599","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:e95d964ef170969342a5c9d45a52aad8532569d2d839f18acfd130ccc124369d","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.369Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryArchiveCacheWorker JID-eda4699bd12b0fdccf4a780d: start","job_status":"start","queue_duration_s":1.740356,"scheduling_latency_s":1.740373} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 435.681980] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryArchiveCacheWorker","jid":"eda4699bd12b0fdccf4a780d","created_at":"2026-09-05T12:01:00.344Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d786b6ca12f5d4297c04819dbf5ae599","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:e95d964ef170969342a5c9d45a52aad8532569d2d839f18acfd130ccc124369d","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.369Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryArchiveCacheWorker JID-eda4699bd12b0fdccf4a780d: done: 0.021895 sec","job_status":"done","queue_duration_s":1.740356,"scheduling_latency_s":1.740373,"redis_calls":12,"redis_duration_s":0.003005,"redis_read_bytes":11,"redis_write_bytes":1392,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001131,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001418,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":206,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000312,"duration_s":0.021895,"completed_at":"2026-09-05T12:01:02.131Z","load_balancing_strategy":"primary","db_duration_s":0.001295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.723334] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"d32dd24ebd55491fbed5d7ae","created_at":"2026-09-05T12:01:00.405Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.424Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-d32dd24ebd55491fbed5d7ae: start","job_status":"start","queue_duration_s":1.708351,"scheduling_latency_s":1.708367} gitlab # [ 435.735977] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"d32dd24ebd55491fbed5d7ae","created_at":"2026-09-05T12:01:00.405Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.424Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-d32dd24ebd55491fbed5d7ae: done: 0.097763 sec","job_status":"done","queue_duration_s":1.708351,"scheduling_latency_s":1.708367,"redis_calls":65,"redis_duration_s":0.015897,"redis_read_bytes":114,"redis_write_bytes":41282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000968,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.000681,"redis_queues_read_bytes":3,"redis_queues_write_bytes":36985,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013533,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031198,"duration_s":0.097763,"completed_at":"2026-09-05T12:01:02.230Z","load_balancing_strategy":"primary","db_duration_s":0.0011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.777419] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.232Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2b65214b2a322f908816b0cc","created_at":"2026-09-05T12:01:00.447Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5101d0cb20582fc814f37f16b4c2d212","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.472Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2b65214b2a322f908816b0cc: start","job_status":"start","queue_duration_s":1.760396,"scheduling_latency_s":1.760414} gitlab # [ 435.791180] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.270Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2b65214b2a322f908816b0cc","created_at":"2026-09-05T12:01:00.447Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5101d0cb20582fc814f37f16b4c2d212","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.472Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2b65214b2a322f908816b0cc: done: 0.038197 sec","job_status":"done","queue_duration_s":1.760396,"scheduling_latency_s":1.760414,"redis_calls":18,"redis_duration_s":0.004909,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001645,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000297,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002156,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000811,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038197,"completed_at":"2026-09-05T12:01:02.270Z","load_balancing_strategy":"primary","db_duration_s":0.003263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.834040] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.272Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"124784353e250059dfaaeaae","created_at":"2026-09-05T12:01:00.483Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac8208d19fbff13c3f11e9335819bf77","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.497Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-124784353e250059dfaaeaae: start","job_status":"start","queue_duration_s":1.774872,"scheduling_latency_s":1.774888} gitlab # [ 435.848694] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.300Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"124784353e250059dfaaeaae","created_at":"2026-09-05T12:01:00.483Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac8208d19fbff13c3f11e9335819bf77","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.497Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-124784353e250059dfaaeaae: done: 0.02849 sec","job_status":"done","queue_duration_s":1.774872,"scheduling_latency_s":1.774888,"redis_calls":11,"redis_duration_s":0.002882,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000476,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001244,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001162,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053978,"exclusive_lock_hold_duration_s":0.00582864,"duration_s":0.02849,"completed_at":"2026-09-05T12:01:02.300Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.891434] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"46adb467f7e21e15e41868f6","created_at":"2026-09-05T12:01:00.505Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c756e1d5ee027e1acd0251758d2e2276","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.519Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-46adb467f7e21e15e41868f6: start","job_status":"start","queue_duration_s":1.782839,"scheduling_latency_s":1.782856} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.23 seconds) gitlab # [ 435.906137] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"46adb467f7e21e15e41868f6","created_at":"2026-09-05T12:01:00.505Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c756e1d5ee027e1acd0251758d2e2276","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.519Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-46adb467f7e21e15e41868f6: done: 0.079499 sec","job_status":"done","queue_duration_s":1.782839,"scheduling_latency_s":1.782856,"redis_calls":13,"redis_duration_s":0.004337000000000001,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001092,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001642,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001603,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053434,"exclusive_lock_hold_duration_s":0.047687,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.079499,"completed_at":"2026-09-05T12:01:02.381Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.950029] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9c5da6b3f56f97a7ce7424f1","created_at":"2026-09-05T12:01:00.533Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fca591c3a1ccdd4f141abc7751dc197","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-05T12:01:00.545Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9c5da6b3f56f97a7ce7424f1: start","job_status":"start","queue_duration_s":1.838779,"scheduling_latency_s":1.838805} gitlab # [ 435.963739] gitaly[897]: time="2026-09-05T12:01:02.776Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ33PVA6ERMY5D0QBB1HQN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:32.174" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:01:02.775" grpc.time_ms=1.096 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.976820] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 128 gitlab # [ 435.979377] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9c5da6b3f56f97a7ce7424f1","created_at":"2026-09-05T12:01:00.533Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fca591c3a1ccdd4f141abc7751dc197","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-05T12:01:00.545Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9c5da6b3f56f97a7ce7424f1: done: 0.046742 sec","job_status":"done","queue_duration_s":1.838779,"scheduling_latency_s":1.838805,"redis_calls":9,"redis_duration_s":0.002709,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001522,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000575,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046742,"completed_at":"2026-09-05T12:01:02.430Z","load_balancing_strategy":"primary","db_duration_s":0.007086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.020434] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.433Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"92552a933de2c9dbb3d86d32","created_at":"2026-09-05T12:01:00.552Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"928849b5c999ebc4214f1b45ab2300e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.577Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-92552a933de2c9dbb3d86d32: start","job_status":"start","queue_duration_s":1.855483,"scheduling_latency_s":1.855505} gitlab # [ 436.034651] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"92552a933de2c9dbb3d86d32","created_at":"2026-09-05T12:01:00.552Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"928849b5c999ebc4214f1b45ab2300e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.577Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-92552a933de2c9dbb3d86d32: done: 0.174408 sec","job_status":"done","queue_duration_s":1.855483,"scheduling_latency_s":1.855505,"redis_calls":12,"redis_duration_s":0.003692,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000577,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002044,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.159074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057426,"exclusive_lock_hold_duration_s":0.15183674,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.174408,"completed_at":"2026-09-05T12:01:02.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008864,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} gitlab # [ 436.079077] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"d7e6f454bd3c484a777f886e","created_at":"2026-09-05T12:01:00.611Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab8bdeb491ce3108d6904af666a9ccaa","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.627Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-d7e6f454bd3c484a777f886e: start","job_status":"start","queue_duration_s":1.98136,"scheduling_latency_s":1.981376} gitlab # [ 436.092795] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"d7e6f454bd3c484a777f886e","created_at":"2026-09-05T12:01:00.611Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab8bdeb491ce3108d6904af666a9ccaa","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.627Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-d7e6f454bd3c484a777f886e: done: 0.020792 sec","job_status":"done","queue_duration_s":1.98136,"scheduling_latency_s":1.981376,"redis_calls":10,"redis_duration_s":0.002275,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031138,"duration_s":0.020792,"completed_at":"2026-09-05T12:01:02.629Z","load_balancing_strategy":"primary","db_duration_s":0.002638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.133910] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"dbd3cbe664e68428ff875783","created_at":"2026-09-05T12:01:00.652Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dba06f0877a4bc9122e11f600c5ed586","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-05T12:01:00.679Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-dbd3cbe664e68428ff875783: start","job_status":"start","queue_duration_s":1.951327,"scheduling_latency_s":1.951342} gitlab # [ 436.147414] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.645Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 436.151198] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"dbd3cbe664e68428ff875783","created_at":"2026-09-05T12:01:00.652Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dba06f0877a4bc9122e11f600c5ed586","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-05T12:01:00.679Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-dbd3cbe664e68428ff875783: done: 0.021375 sec","job_status":"done","queue_duration_s":1.951327,"scheduling_latency_s":1.951342,"redis_calls":9,"redis_duration_s":0.0032069999999999998,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000743,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001866,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000598,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021375,"completed_at":"2026-09-05T12:01:02.652Z","load_balancing_strategy":"primary","db_duration_s":0.002332,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.190812] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"5438dec692080eceba1dbb4b","created_at":"2026-09-05T12:01:00.710Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"484bc6678ce15d5033c4d3811fef0716","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.728Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-5438dec692080eceba1dbb4b: start","job_status":"start","queue_duration_s":1.927388,"scheduling_latency_s":1.927406} gitlab # [ 436.205199] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.674Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"5438dec692080eceba1dbb4b","created_at":"2026-09-05T12:01:00.710Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"484bc6678ce15d5033c4d3811fef0716","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.728Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-5438dec692080eceba1dbb4b: done: 0.018516 sec","job_status":"done","queue_duration_s":1.927388,"scheduling_latency_s":1.927406,"redis_calls":9,"redis_duration_s":0.0012799999999999999,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018516,"completed_at":"2026-09-05T12:01:02.674Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.246402] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e3a98aed1e79f8681c5b3470","created_at":"2026-09-05T12:01:00.744Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60bb263df6da911857b44c1ab606a32d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.758Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e3a98aed1e79f8681c5b3470: start","job_status":"start","queue_duration_s":1.917595,"scheduling_latency_s":1.917613} gitlab # [ 436.259656] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e3a98aed1e79f8681c5b3470","created_at":"2026-09-05T12:01:00.744Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60bb263df6da911857b44c1ab606a32d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.758Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e3a98aed1e79f8681c5b3470: done: 0.101776 sec","job_status":"done","queue_duration_s":1.917595,"scheduling_latency_s":1.917613,"redis_calls":70,"redis_duration_s":0.010797,"redis_read_bytes":119,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001313,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000458,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.008596,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.101776,"completed_at":"2026-09-05T12:01:02.777Z","load_balancing_strategy":"primary","db_duration_s":0.003384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.302250] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:02.782Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"7f3086051a1179e3b00cb0e8","created_at":"2026-09-05T12:01:00.768Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26cd3751656311ee1774737211b85458","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.795Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-7f3086051a1179e3b00cb0e8: start","job_status":"start","queue_duration_s":1.987179,"scheduling_latency_s":1.987199} gitlab # [ 436.315945] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"7f3086051a1179e3b00cb0e8","created_at":"2026-09-05T12:01:00.768Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26cd3751656311ee1774737211b85458","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.795Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-7f3086051a1179e3b00cb0e8: done: 0.249311 sec","job_status":"done","queue_duration_s":1.987179,"scheduling_latency_s":1.987199,"redis_calls":11,"redis_duration_s":0.005176,"redis_read_bytes":9,"redis_write_bytes":1292,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000324,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003584,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.053,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.175384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005886,"exclusive_lock_hold_duration_s":0.22646032,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.249311,"completed_at":"2026-09-05T12:01:03.031Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.07869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.359472] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3127c5e6d1f9e4526036e7c1","created_at":"2026-09-05T12:01:00.833Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f510c6481bebe0e1c49e9f083907b64d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.859Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3127c5e6d1f9e4526036e7c1: start","job_status":"start","queue_duration_s":2.173832,"scheduling_latency_s":2.173848} gitlab # [ 436.374228] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.044Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 436.377676] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3127c5e6d1f9e4526036e7c1","created_at":"2026-09-05T12:01:00.833Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f510c6481bebe0e1c49e9f083907b64d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.859Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3127c5e6d1f9e4526036e7c1: done: 0.018693 sec","job_status":"done","queue_duration_s":2.173832,"scheduling_latency_s":2.173848,"redis_calls":9,"redis_duration_s":0.002787,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001337,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018693,"completed_at":"2026-09-05T12:01:03.052Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.420165] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"141f560e5144bcffb8a7b613","created_at":"2026-09-05T12:01:00.886Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd45a1b0428767e0877c30a77ac18b93","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.900Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-141f560e5144bcffb8a7b613: start","job_status":"start","queue_duration_s":2.152712,"scheduling_latency_s":2.15273} gitlab # [ 436.434335] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"141f560e5144bcffb8a7b613","created_at":"2026-09-05T12:01:00.886Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd45a1b0428767e0877c30a77ac18b93","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.900Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-141f560e5144bcffb8a7b613: done: 0.018568 sec","job_status":"done","queue_duration_s":2.152712,"scheduling_latency_s":2.15273,"redis_calls":11,"redis_duration_s":0.002533,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000517,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000242,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031028,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.018568,"completed_at":"2026-09-05T12:01:03.072Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.478470] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.073Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"43f7705170020ea29b356edd","created_at":"2026-09-05T12:01:00.909Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ceb95d0db12970e655b5e5ba0fe42cbd","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.935Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-43f7705170020ea29b356edd: start","job_status":"start","queue_duration_s":2.138336,"scheduling_latency_s":2.138352} gitlab # [ 436.491777] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.091Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"43f7705170020ea29b356edd","created_at":"2026-09-05T12:01:00.909Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ceb95d0db12970e655b5e5ba0fe42cbd","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.935Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-43f7705170020ea29b356edd: done: 0.018244 sec","job_status":"done","queue_duration_s":2.138336,"scheduling_latency_s":2.138352,"redis_calls":11,"redis_duration_s":0.0026030000000000003,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000465,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018244,"completed_at":"2026-09-05T12:01:03.091Z","load_balancing_strategy":"primary","db_duration_s":0.000862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.535436] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.093Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ImportExportProjectCleanupWorker","jid":"3d00dbac2d17cd9cf0c8491b","created_at":"2026-09-05T12:01:00.961Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec2484edf7159aa3e9db7ef767ab9b48","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c1ab68355154ed2a8f72791841b7337bdce81d68788119e527d7446bb987ef9a","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.990Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ImportExportProjectCleanupWorker JID-3d00dbac2d17cd9cf0c8491b: start","job_status":"start","queue_duration_s":2.102923,"scheduling_latency_s":2.102939} gitlab # [ 436.548836] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.121Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ImportExportProjectCleanupWorker","jid":"3d00dbac2d17cd9cf0c8491b","created_at":"2026-09-05T12:01:00.961Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec2484edf7159aa3e9db7ef767ab9b48","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c1ab68355154ed2a8f72791841b7337bdce81d68788119e527d7446bb987ef9a","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:00.990Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ImportExportProjectCleanupWorker JID-3d00dbac2d17cd9cf0c8491b: done: 0.028096 sec","job_status":"done","queue_duration_s":2.102923,"scheduling_latency_s":2.102939,"redis_calls":12,"redis_duration_s":0.002931,"redis_read_bytes":11,"redis_write_bytes":1420,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001096,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":638,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030886,"duration_s":0.028096,"completed_at":"2026-09-05T12:01:03.121Z","load_balancing_strategy":"primary","db_duration_s":0.005204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.589567] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.122Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"178242e0c801826f17d24d81","created_at":"2026-09-05T12:01:01.003Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4b49a3d8d2b83669a81cb7cb119b965","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-05T12:01:01.040Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-178242e0c801826f17d24d81: start","job_status":"start","queue_duration_s":2.082246,"scheduling_latency_s":2.082263} gitlab # [ 436.602910] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.133Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 436.606305] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"178242e0c801826f17d24d81","created_at":"2026-09-05T12:01:01.003Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4b49a3d8d2b83669a81cb7cb119b965","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-05T12:01:01.040Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-178242e0c801826f17d24d81: done: 0.017824 sec","job_status":"done","queue_duration_s":2.082246,"scheduling_latency_s":2.082263,"redis_calls":9,"redis_duration_s":0.002242,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000512,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017824,"completed_at":"2026-09-05T12:01:03.140Z","load_balancing_strategy":"primary","db_duration_s":0.00064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.646678] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.142Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckCiJobsWorker","jid":"f2ea678bb8b344b5c579b37f","created_at":"2026-09-05T12:01:01.064Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"174d935c689a71dadf8ad30837f5b3d8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:688518068b75df5544ce76496372eb2176b68f67fc20ec19a768c0a774774cd9","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.081Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckCiJobsWorker JID-f2ea678bb8b344b5c579b37f: start","job_status":"start","queue_duration_s":2.060258,"scheduling_latency_s":2.060275} gitlab # [ 436.659890] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.196Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckCiJobsWorker","jid":"f2ea678bb8b344b5c579b37f","created_at":"2026-09-05T12:01:01.064Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"174d935c689a71dadf8ad30837f5b3d8","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:688518068b75df5544ce76496372eb2176b68f67fc20ec19a768c0a774774cd9","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.081Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckCiJobsWorker JID-f2ea678bb8b344b5c579b37f: done: 0.054749 sec","job_status":"done","queue_duration_s":2.060258,"scheduling_latency_s":2.060275,"redis_calls":21,"redis_duration_s":0.006967,"redis_read_bytes":20,"redis_write_bytes":4656,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002656,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1380,"redis_queues_calls":3,"redis_queues_duration_s":0.001264,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2313,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001912,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":502,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001135,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":461,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01799936,"duration_s":0.054749,"completed_at":"2026-09-05T12:01:03.196Z","load_balancing_strategy":"primary","db_duration_s":0.011999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.702782] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"57f9e74117ebfd06590c6d86","created_at":"2026-09-05T12:01:01.093Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"016d3a62fc9be9d45ed51b2458678131","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.107Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-57f9e74117ebfd06590c6d86: start","job_status":"start","queue_duration_s":2.090763,"scheduling_latency_s":2.09078} gitlab # [ 436.717324] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"57f9e74117ebfd06590c6d86","created_at":"2026-09-05T12:01:01.093Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"016d3a62fc9be9d45ed51b2458678131","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.107Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-57f9e74117ebfd06590c6d86: done: 0.027597 sec","job_status":"done","queue_duration_s":2.090763,"scheduling_latency_s":2.09078,"redis_calls":9,"redis_duration_s":0.002391,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001357,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027597,"completed_at":"2026-09-05T12:01:03.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.759059] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"0ffc0ab8eea7acafff3342a6","created_at":"2026-09-05T12:01:01.116Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f9f57349af9d98a83852f09df0b42be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.141Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-0ffc0ab8eea7acafff3342a6: start","job_status":"start","queue_duration_s":2.085339,"scheduling_latency_s":2.085355} gitlab # [ 436.773184] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"0ffc0ab8eea7acafff3342a6","created_at":"2026-09-05T12:01:01.116Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-e8c85004b8e9426a","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f9f57349af9d98a83852f09df0b42be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.141Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-0ffc0ab8eea7acafff3342a6: done: 0.021698 sec","job_status":"done","queue_duration_s":2.085339,"scheduling_latency_s":2.085355,"redis_calls":12,"redis_duration_s":0.005422,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003554,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003037,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.021698,"completed_at":"2026-09-05T12:01:03.248Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.817257] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.250Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"5f5504ea5b54b02e057eb5df","created_at":"2026-09-05T12:01:01.178Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-834f9f0c813947ca","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"0be40828f2bbe6debe26085cc271bd16","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.202Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-5f5504ea5b54b02e057eb5df: start","job_status":"start","queue_duration_s":2.047366,"scheduling_latency_s":2.047382} gitlab # [ 436.831975] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.271Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"5f5504ea5b54b02e057eb5df","created_at":"2026-09-05T12:01:01.178Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-834f9f0c813947ca","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"0be40828f2bbe6debe26085cc271bd16","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.202Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-5f5504ea5b54b02e057eb5df: done: 0.021139 sec","job_status":"done","queue_duration_s":2.047366,"scheduling_latency_s":2.047382,"redis_calls":10,"redis_duration_s":0.002341,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000481,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001382,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000478,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030724,"duration_s":0.021139,"completed_at":"2026-09-05T12:01:03.271Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.873603] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.272Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"bdaa0ddaad03e3c3a238a158","created_at":"2026-09-05T12:01:01.204Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-834f9f0c813947ca","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"0be40828f2bbe6debe26085cc271bd16","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.216Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-bdaa0ddaad03e3c3a238a158: start","job_status":"start","queue_duration_s":2.05647,"scheduling_latency_s":2.056486} gitlab # [ 436.888129] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.293Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"bdaa0ddaad03e3c3a238a158","created_at":"2026-09-05T12:01:01.204Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-834f9f0c813947ca","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"0be40828f2bbe6debe26085cc271bd16","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.216Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-bdaa0ddaad03e3c3a238a158: done: 0.020848 sec","job_status":"done","queue_duration_s":2.05647,"scheduling_latency_s":2.056486,"redis_calls":10,"redis_duration_s":0.002477,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000525,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001429,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000523,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030454,"duration_s":0.020848,"completed_at":"2026-09-05T12:01:03.293Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.931186] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"dc202993fcb88d890fe81809","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-dc202993fcb88d890fe81809: start","job_status":"start","queue_duration_s":1.333969,"scheduling_latency_s":1.333983} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 436.943420] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"dc202993fcb88d890fe81809","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-dc202993fcb88d890fe81809: done: 0.031982 sec","job_status":"done","queue_duration_s":1.333969,"scheduling_latency_s":1.333983,"redis_calls":18,"redis_duration_s":0.00445,"redis_read_bytes":39,"redis_write_bytes":2116,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001277,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":471,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002705,"redis_shared_state_read_bytes":34,"redis_shared_state_write_bytes":1324,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044188,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.031982,"completed_at":"2026-09-05T12:01:03.326Z","load_balancing_strategy":"primary","db_duration_s":0.005327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.983043] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"594b0d73512ffaa5b327e070","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-594b0d73512ffaa5b327e070: start","job_status":"start","queue_duration_s":1.367347,"scheduling_latency_s":1.367363} gitlab # [ 436.995242] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"594b0d73512ffaa5b327e070","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-594b0d73512ffaa5b327e070: done: 0.02426 sec","job_status":"done","queue_duration_s":1.367347,"scheduling_latency_s":1.367363,"redis_calls":15,"redis_duration_s":0.003621,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002124,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.02426,"completed_at":"2026-09-05T12:01:03.352Z","load_balancing_strategy":"primary","db_duration_s":0.003952,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.036132] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"91178930cd5b49d92129f66a","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-91178930cd5b49d92129f66a: start","job_status":"start","queue_duration_s":1.393281,"scheduling_latency_s":1.393299} gitlab # [ 437.048462] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"91178930cd5b49d92129f66a","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-91178930cd5b49d92129f66a: done: 0.023601 sec","job_status":"done","queue_duration_s":1.393281,"scheduling_latency_s":1.393299,"redis_calls":15,"redis_duration_s":0.0031219999999999998,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.023601,"completed_at":"2026-09-05T12:01:03.377Z","load_balancing_strategy":"primary","db_duration_s":0.004106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.089335] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"f4985a2b6de4640a305a000c","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-f4985a2b6de4640a305a000c: start","job_status":"start","queue_duration_s":1.418404,"scheduling_latency_s":1.418421} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.18 seconds) gitlab # [ 437.101598] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"f4985a2b6de4640a305a000c","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-f4985a2b6de4640a305a000c: done: 0.024368 sec","job_status":"done","queue_duration_s":1.418404,"scheduling_latency_s":1.418421,"redis_calls":15,"redis_duration_s":0.003371,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002308,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":1,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":1,"duration_s":0.024368,"completed_at":"2026-09-05T12:01:03.403Z","load_balancing_strategy":"primary","db_duration_s":0.004297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.143257] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"34d1476b7e29844a68044432","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-34d1476b7e29844a68044432: start","job_status":"start","queue_duration_s":1.444129,"scheduling_latency_s":1.444145} gitlab # [ 437.155943] gitaly[897]: time="2026-09-05T12:01:03.966Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ34X69TTQAW31XBZMQF3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:33.065" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:01:03.965" grpc.time_ms=0.584 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.168627] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 80 gitlab # [ 437.171100] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"34d1476b7e29844a68044432","created_at":"2026-09-05T12:01:01.946Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-3bcd42d56fab4f1b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"720a6e4bf8224d1d6bec6f43eca61ee5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:01.960Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-34d1476b7e29844a68044432: done: 0.02399 sec","job_status":"done","queue_duration_s":1.444129,"scheduling_latency_s":1.444145,"redis_calls":15,"redis_duration_s":0.003239,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.00222,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":1,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":1,"duration_s":0.02399,"completed_at":"2026-09-05T12:01:03.429Z","load_balancing_strategy":"primary","db_duration_s":0.004262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.211978] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f43ab22a4849ed29e3601874","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f43ab22a4849ed29e3601874: start","job_status":"start","queue_duration_s":1.393587,"scheduling_latency_s":1.393604} gitlab # [ 437.225087] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f43ab22a4849ed29e3601874","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f43ab22a4849ed29e3601874: done: 0.03653 sec","job_status":"done","queue_duration_s":1.393587,"scheduling_latency_s":1.393604,"redis_calls":16,"redis_duration_s":0.003886,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002269,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031014,"duration_s":0.03653,"completed_at":"2026-09-05T12:01:03.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.264762] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c30f6900257b96b6af5a44de","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c30f6900257b96b6af5a44de: start","job_status":"start","queue_duration_s":1.431466,"scheduling_latency_s":1.431482} gitlab # [ 437.277673] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c30f6900257b96b6af5a44de","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c30f6900257b96b6af5a44de: done: 0.016991 sec","job_status":"done","queue_duration_s":1.431466,"scheduling_latency_s":1.431482,"redis_calls":13,"redis_duration_s":0.00285,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016991,"completed_at":"2026-09-05T12:01:03.485Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.316207] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8e90a61152c45c2f477049fd","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8e90a61152c45c2f477049fd: start","job_status":"start","queue_duration_s":1.449865,"scheduling_latency_s":1.449882} gitlab # [ 437.329355] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8e90a61152c45c2f477049fd","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8e90a61152c45c2f477049fd: done: 0.017017 sec","job_status":"done","queue_duration_s":1.449865,"scheduling_latency_s":1.449882,"redis_calls":13,"redis_duration_s":0.002913,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017017,"completed_at":"2026-09-05T12:01:03.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001649,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.369263] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d00e37baefa0251df1abd284","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d00e37baefa0251df1abd284: start","job_status":"start","queue_duration_s":1.468222,"scheduling_latency_s":1.468238} gitlab # [ 437.382385] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d00e37baefa0251df1abd284","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d00e37baefa0251df1abd284: done: 0.016573 sec","job_status":"done","queue_duration_s":1.468222,"scheduling_latency_s":1.468238,"redis_calls":13,"redis_duration_s":0.002653,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016573,"completed_at":"2026-09-05T12:01:03.521Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.421307] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c86df7ba253eb5e80b437c2c","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c86df7ba253eb5e80b437c2c: start","job_status":"start","queue_duration_s":1.486223,"scheduling_latency_s":1.486239} gitlab # [ 437.434177] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c86df7ba253eb5e80b437c2c","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c86df7ba253eb5e80b437c2c: done: 0.016619 sec","job_status":"done","queue_duration_s":1.486223,"scheduling_latency_s":1.486239,"redis_calls":13,"redis_duration_s":0.002794,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016619,"completed_at":"2026-09-05T12:01:03.539Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.473162] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e0858da4c263b73e86638cca","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e0858da4c263b73e86638cca: start","job_status":"start","queue_duration_s":1.504285,"scheduling_latency_s":1.504301} gitlab # [ 437.486329] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e0858da4c263b73e86638cca","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e0858da4c263b73e86638cca: done: 0.016818 sec","job_status":"done","queue_duration_s":1.504285,"scheduling_latency_s":1.504301,"redis_calls":13,"redis_duration_s":0.002971,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00205,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016818,"completed_at":"2026-09-05T12:01:03.558Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.525317] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8609a28f0ebaea009faf7f5f","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8609a28f0ebaea009faf7f5f: start","job_status":"start","queue_duration_s":1.522317,"scheduling_latency_s":1.522333} gitlab # [ 437.538333] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8609a28f0ebaea009faf7f5f","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8609a28f0ebaea009faf7f5f: done: 0.016097 sec","job_status":"done","queue_duration_s":1.522317,"scheduling_latency_s":1.522333,"redis_calls":13,"redis_duration_s":0.002918,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016097,"completed_at":"2026-09-05T12:01:03.575Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.578127] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"140b4ffb631ae2fff53b21cf","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-140b4ffb631ae2fff53b21cf: start","job_status":"start","queue_duration_s":1.539873,"scheduling_latency_s":1.53989} gitlab # [ 437.591474] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"140b4ffb631ae2fff53b21cf","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-140b4ffb631ae2fff53b21cf: done: 0.036891 sec","job_status":"done","queue_duration_s":1.539873,"scheduling_latency_s":1.53989,"redis_calls":13,"redis_duration_s":0.0037470000000000003,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001129,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002618,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036891,"completed_at":"2026-09-05T12:01:03.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.631013] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"786a4109128d3f92832e264d","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-786a4109128d3f92832e264d: start","job_status":"start","queue_duration_s":1.579595,"scheduling_latency_s":1.579634} gitlab # [ 437.644313] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"786a4109128d3f92832e264d","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-786a4109128d3f92832e264d: done: 0.037946 sec","job_status":"done","queue_duration_s":1.579595,"scheduling_latency_s":1.579634,"redis_calls":13,"redis_duration_s":0.004844,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002422,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002422,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037946,"completed_at":"2026-09-05T12:01:03.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.683574] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"430d417db2f7230acd0f8f47","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-430d417db2f7230acd0f8f47: start","job_status":"start","queue_duration_s":1.620569,"scheduling_latency_s":1.620609} gitlab # [ 437.696779] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"430d417db2f7230acd0f8f47","created_at":"2026-09-05T12:01:02.016Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-35948dcc82334c38","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"57e7d480d709ae409eff014c93346ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.036Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-430d417db2f7230acd0f8f47: done: 0.031511 sec","job_status":"done","queue_duration_s":1.620569,"scheduling_latency_s":1.620609,"redis_calls":13,"redis_duration_s":0.0037159999999999997,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002492,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031511,"completed_at":"2026-09-05T12:01:03.688Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.736431] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fc6a59b2818097ac1db28484","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fc6a59b2818097ac1db28484: start","job_status":"start","queue_duration_s":1.469775,"scheduling_latency_s":1.469803} gitlab # [ 437.748655] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fc6a59b2818097ac1db28484","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fc6a59b2818097ac1db28484: done: 0.048851 sec","job_status":"done","queue_duration_s":1.469775,"scheduling_latency_s":1.469803,"redis_calls":13,"redis_duration_s":0.003962,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00068,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001393,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001889,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045868,"duration_s":0.048851,"completed_at":"2026-09-05T12:01:03.739Z","load_balancing_strategy":"primary","db_duration_s":0.006956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.789459] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ce186fba5b5b7fa57299898b","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ce186fba5b5b7fa57299898b: start","job_status":"start","queue_duration_s":1.520566,"scheduling_latency_s":1.52059} gitlab # [ 437.801597] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ce186fba5b5b7fa57299898b","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ce186fba5b5b7fa57299898b: done: 0.026562 sec","job_status":"done","queue_duration_s":1.520566,"scheduling_latency_s":1.52059,"redis_calls":10,"redis_duration_s":0.002805,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001272,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001533,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026562,"completed_at":"2026-09-05T12:01:03.768Z","load_balancing_strategy":"primary","db_duration_s":0.002655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.839824] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4acabda176f805cdaac7cf10","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4acabda176f805cdaac7cf10: start","job_status":"start","queue_duration_s":1.548839,"scheduling_latency_s":1.548863} gitlab # [ 437.852406] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4acabda176f805cdaac7cf10","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4acabda176f805cdaac7cf10: done: 0.024707 sec","job_status":"done","queue_duration_s":1.548839,"scheduling_latency_s":1.548863,"redis_calls":10,"redis_duration_s":0.002656,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001597,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024707,"completed_at":"2026-09-05T12:01:03.794Z","load_balancing_strategy":"primary","db_duration_s":0.00257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.890861] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4dfa9c2e7ce1a87b336246cf","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4dfa9c2e7ce1a87b336246cf: start","job_status":"start","queue_duration_s":1.575063,"scheduling_latency_s":1.575083} gitlab # [ 437.903088] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4dfa9c2e7ce1a87b336246cf","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4dfa9c2e7ce1a87b336246cf: done: 0.02278 sec","job_status":"done","queue_duration_s":1.575063,"scheduling_latency_s":1.575083,"redis_calls":10,"redis_duration_s":0.002516,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001478,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02278,"completed_at":"2026-09-05T12:01:03.819Z","load_balancing_strategy":"primary","db_duration_s":0.002644,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.941538] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"90ae4dd3b1f68ec1b38ec82a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-90ae4dd3b1f68ec1b38ec82a: start","job_status":"start","queue_duration_s":1.59933,"scheduling_latency_s":1.59935} gitlab # [ 437.953794] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"90ae4dd3b1f68ec1b38ec82a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-90ae4dd3b1f68ec1b38ec82a: done: 0.021059 sec","job_status":"done","queue_duration_s":1.59933,"scheduling_latency_s":1.59935,"redis_calls":10,"redis_duration_s":0.0025859999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001555,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021059,"completed_at":"2026-09-05T12:01:03.841Z","load_balancing_strategy":"primary","db_duration_s":0.002497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.993062] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"60441aae22b0d61e11f2417a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-60441aae22b0d61e11f2417a: start","job_status":"start","queue_duration_s":1.621981,"scheduling_latency_s":1.622001} gitlab # [ 438.005632] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"60441aae22b0d61e11f2417a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-60441aae22b0d61e11f2417a: done: 0.020292 sec","job_status":"done","queue_duration_s":1.621981,"scheduling_latency_s":1.622001,"redis_calls":10,"redis_duration_s":0.0023870000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001346,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020292,"completed_at":"2026-09-05T12:01:03.863Z","load_balancing_strategy":"primary","db_duration_s":0.002563,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.044049] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"620a730698a01f88f2695268","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-620a730698a01f88f2695268: start","job_status":"start","queue_duration_s":1.643722,"scheduling_latency_s":1.643739} gitlab # [ 438.056169] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"620a730698a01f88f2695268","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-620a730698a01f88f2695268: done: 0.049481 sec","job_status":"done","queue_duration_s":1.643722,"scheduling_latency_s":1.643739,"redis_calls":10,"redis_duration_s":0.003898,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001261,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002637,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049481,"completed_at":"2026-09-05T12:01:03.914Z","load_balancing_strategy":"primary","db_duration_s":0.0037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.094649] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8b7c4ff3085f1e9d079126ce","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8b7c4ff3085f1e9d079126ce: start","job_status":"start","queue_duration_s":1.696727,"scheduling_latency_s":1.696773} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 438.106969] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8b7c4ff3085f1e9d079126ce","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8b7c4ff3085f1e9d079126ce: done: 0.043763 sec","job_status":"done","queue_duration_s":1.696727,"scheduling_latency_s":1.696773,"redis_calls":10,"redis_duration_s":0.003152,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043763,"completed_at":"2026-09-05T12:01:03.961Z","load_balancing_strategy":"primary","db_duration_s":0.004374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.143847] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5b7c0b5c4fff362b484aec57","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5b7c0b5c4fff362b484aec57: start","job_status":"start","queue_duration_s":1.742806,"scheduling_latency_s":1.742841} gitlab # [ 438.155446] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5b7c0b5c4fff362b484aec57","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5b7c0b5c4fff362b484aec57: done: 0.033648 sec","job_status":"done","queue_duration_s":1.742806,"scheduling_latency_s":1.742841,"redis_calls":10,"redis_duration_s":0.002691,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001643,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033648,"completed_at":"2026-09-05T12:01:03.997Z","load_balancing_strategy":"primary","db_duration_s":0.003325,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.195853] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:03.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"81161e1f3f5ef75c02786448","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-81161e1f3f5ef75c02786448: start","job_status":"start","queue_duration_s":1.778074,"scheduling_latency_s":1.778098} gitlab # [ 438.208642] nix-daemon[3765]: accepted connection from pid 4472, user nixbot gitlab # [ 438.209601] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"81161e1f3f5ef75c02786448","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-81161e1f3f5ef75c02786448: done: 0.02957 sec","job_status":"done","queue_duration_s":1.778074,"scheduling_latency_s":1.778098,"redis_calls":10,"redis_duration_s":0.0029059999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001211,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001695,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02957,"completed_at":"2026-09-05T12:01:04.028Z","load_balancing_strategy":"primary","db_duration_s":0.00293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.248024] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b06ee8c107323b76e33f12fe","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b06ee8c107323b76e33f12fe: start","job_status":"start","queue_duration_s":1.809605,"scheduling_latency_s":1.80963} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.18 seconds) gitlab # [ 438.259997] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b06ee8c107323b76e33f12fe","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b06ee8c107323b76e33f12fe: done: 0.025077 sec","job_status":"done","queue_duration_s":1.809605,"scheduling_latency_s":1.80963,"redis_calls":10,"redis_duration_s":0.002703,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001549,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025077,"completed_at":"2026-09-05T12:01:04.055Z","load_balancing_strategy":"primary","db_duration_s":0.002576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.297877] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f67fc8fa0d380041cefd3aed","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f67fc8fa0d380041cefd3aed: start","job_status":"start","queue_duration_s":1.836353,"scheduling_latency_s":1.836401} gitlab # [ 438.310178] gitaly[897]: time="2026-09-05T12:01:05.143Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ361ZJPY12706P6HRN635 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:35.141" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:01:05.142" grpc.time_ms=0.989 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.323181] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 85 gitlab # [ 438.325878] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f67fc8fa0d380041cefd3aed","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f67fc8fa0d380041cefd3aed: done: 0.023453 sec","job_status":"done","queue_duration_s":1.836353,"scheduling_latency_s":1.836401,"redis_calls":10,"redis_duration_s":0.0026550000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001112,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023453,"completed_at":"2026-09-05T12:01:04.081Z","load_balancing_strategy":"primary","db_duration_s":0.002627,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.364061] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c15a7a6c58b1f76f4214aafc","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c15a7a6c58b1f76f4214aafc: start","job_status":"start","queue_duration_s":1.861411,"scheduling_latency_s":1.861431} gitlab # [ 438.375869] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c15a7a6c58b1f76f4214aafc","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c15a7a6c58b1f76f4214aafc: done: 0.023903 sec","job_status":"done","queue_duration_s":1.861411,"scheduling_latency_s":1.861431,"redis_calls":10,"redis_duration_s":0.002686,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001583,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023903,"completed_at":"2026-09-05T12:01:04.106Z","load_balancing_strategy":"primary","db_duration_s":0.002706,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.415351] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6d2177aec15d28818ddc9ad6","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6d2177aec15d28818ddc9ad6: start","job_status":"start","queue_duration_s":1.886895,"scheduling_latency_s":1.886915} gitlab # [ 438.427366] gitaly[897]: time="2026-09-05T12:01:05.232Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ3630E16KP6Q6ZK576Q7X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:35.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:01:05.231" grpc.time_ms=0.605 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.440391] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6d2177aec15d28818ddc9ad6","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6d2177aec15d28818ddc9ad6: done: 0.021235 sec","job_status":"done","queue_duration_s":1.886895,"scheduling_latency_s":1.886915,"redis_calls":10,"redis_duration_s":0.002529,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001527,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021235,"completed_at":"2026-09-05T12:01:04.129Z","load_balancing_strategy":"primary","db_duration_s":0.002686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.478577] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1dcc5404d7c784cc38e17148","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1dcc5404d7c784cc38e17148: start","job_status":"start","queue_duration_s":1.909698,"scheduling_latency_s":1.909716} gitlab # [ 438.490746] gitaly[897]: time="2026-09-05T12:01:05.297Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ3630E16KP6Q6ZK576Q7X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:15.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:01:05.284" grpc.time_ms=12.152 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.504558] nixbot[3535]: {"ts": "2026-09-05T12:01:05.364Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 \"HTTP/1.1 201 Created\""} gitlab # [ 438.507281] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 242 gitlab # [ 438.509915] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1dcc5404d7c784cc38e17148","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1dcc5404d7c784cc38e17148: done: 0.019825 sec","job_status":"done","queue_duration_s":1.909698,"scheduling_latency_s":1.909716,"redis_calls":10,"redis_duration_s":0.00255,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00107,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00148,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019825,"completed_at":"2026-09-05T12:01:04.150Z","load_balancing_strategy":"primary","db_duration_s":0.002656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.547706] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8d85728a21634fd90c6e4215","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8d85728a21634fd90c6e4215: start","job_status":"start","queue_duration_s":1.930896,"scheduling_latency_s":1.930913} gitlab # [ 438.560026] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8d85728a21634fd90c6e4215","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8d85728a21634fd90c6e4215: done: 0.020058 sec","job_status":"done","queue_duration_s":1.930896,"scheduling_latency_s":1.930913,"redis_calls":10,"redis_duration_s":0.002993,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001462,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020058,"completed_at":"2026-09-05T12:01:04.172Z","load_balancing_strategy":"primary","db_duration_s":0.002495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.598123] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"855ef1251d8a880b5458a044","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-855ef1251d8a880b5458a044: start","job_status":"start","queue_duration_s":1.952926,"scheduling_latency_s":1.952975} gitlab # [ 438.610204] gitaly[897]: time="2026-09-05T12:01:05.425Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ36AR42ZEZ9XEK0G2SCPR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:35.223" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:01:05.423" grpc.time_ms=0.855 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.623822] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"855ef1251d8a880b5458a044","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-855ef1251d8a880b5458a044: done: 0.048943 sec","job_status":"done","queue_duration_s":1.952926,"scheduling_latency_s":1.952975,"redis_calls":10,"redis_duration_s":0.004675,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002579,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048943,"completed_at":"2026-09-05T12:01:04.223Z","load_balancing_strategy":"primary","db_duration_s":0.003576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.662427] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"270fd2a548eb22665c960312","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-270fd2a548eb22665c960312: start","job_status":"start","queue_duration_s":2.005072,"scheduling_latency_s":2.005117} gitlab # [ 438.674575] gitaly[897]: time="2026-09-05T12:01:05.472Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ36AR42ZEZ9XEK0G2SCPR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:15.181" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:01:05.463" grpc.time_ms=9.253 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.687971] nixbot[3535]: {"ts": "2026-09-05T12:01:05.528Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 \"HTTP/1.1 201 Created\""} gitlab # [ 438.690853] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 157 gitlab # [ 438.693410] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"270fd2a548eb22665c960312","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-270fd2a548eb22665c960312: done: 0.040418 sec","job_status":"done","queue_duration_s":2.005072,"scheduling_latency_s":2.005117,"redis_calls":10,"redis_duration_s":0.003392,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001449,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001943,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040418,"completed_at":"2026-09-05T12:01:04.266Z","load_balancing_strategy":"primary","db_duration_s":0.003181,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.731029] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"50c6d68255911aa3cd04a2ee","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-50c6d68255911aa3cd04a2ee: start","job_status":"start","queue_duration_s":2.047824,"scheduling_latency_s":2.047855} gitlab # [ 438.743342] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"50c6d68255911aa3cd04a2ee","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-50c6d68255911aa3cd04a2ee: done: 0.031311 sec","job_status":"done","queue_duration_s":2.047824,"scheduling_latency_s":2.047855,"redis_calls":10,"redis_duration_s":0.00321,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001345,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031311,"completed_at":"2026-09-05T12:01:04.300Z","load_balancing_strategy":"primary","db_duration_s":0.002987,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.781434] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f2ffd3389be7ea726e64a45a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f2ffd3389be7ea726e64a45a: start","job_status":"start","queue_duration_s":2.081174,"scheduling_latency_s":2.081201} gitlab # [ 438.793437] gitaly[897]: time="2026-09-05T12:01:05.594Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ36G15KZP7M3JHVZCT9Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-05T12:01:35.392" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-05T12:01:05.593" grpc.time_ms=1.216 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.805931] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f2ffd3389be7ea726e64a45a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f2ffd3389be7ea726e64a45a: done: 0.028339 sec","job_status":"done","queue_duration_s":2.081174,"scheduling_latency_s":2.081201,"redis_calls":10,"redis_duration_s":0.0030080000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001199,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001809,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028339,"completed_at":"2026-09-05T12:01:04.330Z","load_balancing_strategy":"primary","db_duration_s":0.002935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.845050] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"48507fe86ab74f67dad30337","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-48507fe86ab74f67dad30337: start","job_status":"start","queue_duration_s":2.111411,"scheduling_latency_s":2.111435} gitlab # [ 438.857350] gitaly[897]: time="2026-09-05T12:01:05.642Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ36G15KZP7M3JHVZCT9Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-05T12:01:15.032" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-05T12:01:05.632" grpc.time_ms=9.292 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.871050] nixbot[3535]: {"ts": "2026-09-05T12:01:05.699Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 \"HTTP/1.1 201 Created\""} gitlab # [ 438.873961] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 160 gitlab # [ 438.876513] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"48507fe86ab74f67dad30337","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-48507fe86ab74f67dad30337: done: 0.024866 sec","job_status":"done","queue_duration_s":2.111411,"scheduling_latency_s":2.111435,"redis_calls":10,"redis_duration_s":0.002757,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001291,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001466,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024866,"completed_at":"2026-09-05T12:01:04.357Z","load_balancing_strategy":"primary","db_duration_s":0.00273,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.914729] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"93a117ade8620ea340f3b3c4","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-93a117ade8620ea340f3b3c4: start","job_status":"start","queue_duration_s":2.138823,"scheduling_latency_s":2.138845} gitlab # [ 438.927186] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"93a117ade8620ea340f3b3c4","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-93a117ade8620ea340f3b3c4: done: 0.023729 sec","job_status":"done","queue_duration_s":2.138823,"scheduling_latency_s":2.138845,"redis_calls":10,"redis_duration_s":0.002656,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001516,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023729,"completed_at":"2026-09-05T12:01:04.383Z","load_balancing_strategy":"primary","db_duration_s":0.002663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.965628] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"415e7a47dd9b4770bff61a84","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-415e7a47dd9b4770bff61a84: start","job_status":"start","queue_duration_s":2.164093,"scheduling_latency_s":2.164113} gitlab # [ 438.977979] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"415e7a47dd9b4770bff61a84","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-415e7a47dd9b4770bff61a84: done: 0.185474 sec","job_status":"done","queue_duration_s":2.164093,"scheduling_latency_s":2.164113,"redis_calls":10,"redis_duration_s":0.0030470000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001672,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.175957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.185474,"completed_at":"2026-09-05T12:01:04.570Z","load_balancing_strategy":"primary","db_duration_s":0.003583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.016548] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"77e47e0fbcbac630165c239a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-77e47e0fbcbac630165c239a: start","job_status":"start","queue_duration_s":2.351033,"scheduling_latency_s":2.351052} gitlab # [ 439.028763] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"77e47e0fbcbac630165c239a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-77e47e0fbcbac630165c239a: done: 0.020047 sec","job_status":"done","queue_duration_s":2.351033,"scheduling_latency_s":2.351052,"redis_calls":10,"redis_duration_s":0.002555,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00106,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020047,"completed_at":"2026-09-05T12:01:04.592Z","load_balancing_strategy":"primary","db_duration_s":0.00285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.068107] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e3f4cf774704cb623e80334a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e3f4cf774704cb623e80334a: start","job_status":"start","queue_duration_s":2.372445,"scheduling_latency_s":2.37246} gitlab # [ 439.080324] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e3f4cf774704cb623e80334a","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e3f4cf774704cb623e80334a: done: 0.019921 sec","job_status":"done","queue_duration_s":2.372445,"scheduling_latency_s":2.37246,"redis_calls":10,"redis_duration_s":0.002549,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001491,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019921,"completed_at":"2026-09-05T12:01:04.613Z","load_balancing_strategy":"primary","db_duration_s":0.002888,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.118716] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0601df2712c9717f9f22a0a8","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0601df2712c9717f9f22a0a8: start","job_status":"start","queue_duration_s":2.393746,"scheduling_latency_s":2.393762} gitlab # [ 439.131159] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0601df2712c9717f9f22a0a8","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0601df2712c9717f9f22a0a8: done: 0.02083 sec","job_status":"done","queue_duration_s":2.393746,"scheduling_latency_s":2.393762,"redis_calls":10,"redis_duration_s":0.0025139999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001459,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02083,"completed_at":"2026-09-05T12:01:04.635Z","load_balancing_strategy":"primary","db_duration_s":0.002755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.170191] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6f792650d27d3176cca6d8aa","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6f792650d27d3176cca6d8aa: start","job_status":"start","queue_duration_s":2.415958,"scheduling_latency_s":2.415974} gitlab # [ 439.182667] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6f792650d27d3176cca6d8aa","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6f792650d27d3176cca6d8aa: done: 0.019912 sec","job_status":"done","queue_duration_s":2.415958,"scheduling_latency_s":2.415974,"redis_calls":10,"redis_duration_s":0.0025659999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001483,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019912,"completed_at":"2026-09-05T12:01:04.657Z","load_balancing_strategy":"primary","db_duration_s":0.002675,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.222374] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4e746ad242c059b67637e08b","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4e746ad242c059b67637e08b: start","job_status":"start","queue_duration_s":2.438016,"scheduling_latency_s":2.438037} gitlab # [ 439.234764] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4e746ad242c059b67637e08b","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4e746ad242c059b67637e08b: done: 0.020572 sec","job_status":"done","queue_duration_s":2.438016,"scheduling_latency_s":2.438037,"redis_calls":10,"redis_duration_s":0.002535,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001485,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020572,"completed_at":"2026-09-05T12:01:04.679Z","load_balancing_strategy":"primary","db_duration_s":0.002676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.274708] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4c0f3770db35573f3747fa35","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4c0f3770db35573f3747fa35: start","job_status":"start","queue_duration_s":2.45927,"scheduling_latency_s":2.459286} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses gitlab # [ 439.287019] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4c0f3770db35573f3747fa35","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4c0f3770db35573f3747fa35: done: 0.070136 sec","job_status":"done","queue_duration_s":2.45927,"scheduling_latency_s":2.459286,"redis_calls":10,"redis_duration_s":0.00634,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003727,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.018,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070136,"completed_at":"2026-09-05T12:01:04.750Z","load_balancing_strategy":"primary","db_duration_s":0.025446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.323159] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"50c0ed783e9bcc22ad434a3d","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-50c0ed783e9bcc22ad434a3d: start","job_status":"start","queue_duration_s":2.537228,"scheduling_latency_s":2.537275} gitlab # [ 439.334628] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"50c0ed783e9bcc22ad434a3d","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-50c0ed783e9bcc22ad434a3d: done: 0.104649 sec","job_status":"done","queue_duration_s":2.537228,"scheduling_latency_s":2.537275,"redis_calls":10,"redis_duration_s":0.016941,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006679,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.010262,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.018,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.104649,"completed_at":"2026-09-05T12:01:04.863Z","load_balancing_strategy":"primary","db_duration_s":0.01879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.373217] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:04.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"880c1978657367a0cc2e37de","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-880c1978657367a0cc2e37de: start","job_status":"start","queue_duration_s":2.658125,"scheduling_latency_s":2.658167} gitlab # [ 439.385965] gitaly[897]: time="2026-09-05T12:01:06.189Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ371VBR42A52JGYNMQTD7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:01:06.178" grpc.time_ms=11.358 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 439.399726] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 82 gitlab # [ 439.401978] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"880c1978657367a0cc2e37de","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-880c1978657367a0cc2e37de: done: 0.204907 sec","job_status":"done","queue_duration_s":2.658125,"scheduling_latency_s":2.658167,"redis_calls":10,"redis_duration_s":0.008206999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004263,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.093,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.093,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.204907,"completed_at":"2026-09-05T12:01:05.084Z","load_balancing_strategy":"primary","db_duration_s":0.092364,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.440861] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4c98b6cc94a1c7714c2ab049","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4c98b6cc94a1c7714c2ab049: start","job_status":"start","queue_duration_s":2.869034,"scheduling_latency_s":2.869067} gitlab # [ 439.455654] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4c98b6cc94a1c7714c2ab049","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4c98b6cc94a1c7714c2ab049: done: 0.040389 sec","job_status":"done","queue_duration_s":2.869034,"scheduling_latency_s":2.869067,"redis_calls":10,"redis_duration_s":0.005086,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00208,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003006,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040389,"completed_at":"2026-09-05T12:01:05.130Z","load_balancing_strategy":"primary","db_duration_s":0.008891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.496269] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5380a56ddaf1f15effcb64c1","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5380a56ddaf1f15effcb64c1: start","job_status":"start","queue_duration_s":2.919328,"scheduling_latency_s":2.919364} gitlab # [ 439.508592] gitlab-workhorse[823]: time="2026-09-05T12:01:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 439.510450] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 107 gitlab # [ 439.512437] gitaly[897]: time="2026-09-05T12:01:06.300Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ374JBDEA2XNXTQWZMAG9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:01:06.290" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 439.522397] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5380a56ddaf1f15effcb64c1","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5380a56ddaf1f15effcb64c1: done: 0.031755 sec","job_status":"done","queue_duration_s":2.919328,"scheduling_latency_s":2.919364,"redis_calls":10,"redis_duration_s":0.002248,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031755,"completed_at":"2026-09-05T12:01:05.172Z","load_balancing_strategy":"primary","db_duration_s":0.001897,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.561217] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"872433732c1e23155aed45d4","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-872433732c1e23155aed45d4: start","job_status":"start","queue_duration_s":2.953882,"scheduling_latency_s":2.953904} gitlab # [ 439.573724] gitaly[897]: time="2026-09-05T12:01:06.300Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ374JBDEA2XNXTQWZMAG9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:01:06.290" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 439.587360] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"872433732c1e23155aed45d4","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-872433732c1e23155aed45d4: done: 0.022556 sec","job_status":"done","queue_duration_s":2.953882,"scheduling_latency_s":2.953904,"redis_calls":10,"redis_duration_s":0.001478,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000878,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022556,"completed_at":"2026-09-05T12:01:05.197Z","load_balancing_strategy":"primary","db_duration_s":0.001575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.626369] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9c888f46918d0c7c532e1d06","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9c888f46918d0c7c532e1d06: start","job_status":"start","queue_duration_s":2.977749,"scheduling_latency_s":2.977769} gitlab # [ 439.638500] gitaly[897]: time="2026-09-05T12:01:06.305Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1RQ374JBDEA2XNXTQWZMAG9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:01:06.290" grpc.time_ms=10.684 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 439.653095] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 115 gitlab # [ 439.655510] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9c888f46918d0c7c532e1d06","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9c888f46918d0c7c532e1d06: done: 0.019527 sec","job_status":"done","queue_duration_s":2.977749,"scheduling_latency_s":2.977769,"redis_calls":10,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000835,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019527,"completed_at":"2026-09-05T12:01:05.218Z","load_balancing_strategy":"primary","db_duration_s":0.001283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.693586] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cec5bc1496a5d4965de152cc","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cec5bc1496a5d4965de152cc: start","job_status":"start","queue_duration_s":2.998365,"scheduling_latency_s":2.998382} gitlab # [ 439.706802] gitaly[897]: time="2026-09-05T12:01:06.470Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1RQ379KPRVTYX3W3BXNT18B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-05T12:01:06.458" grpc.time_ms=10.967 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses, in 0.43 seconds) gitlab # [ 439.719887] gitlab-workhorse[823]: time="2026-09-05T12:01:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 (finished: subtest: eval and build post commit statuses, in 10.22 seconds) gitlab # [ 439.721672] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 104 (finished: run the VM test script, in 509.33 seconds) gitlab # [ 439.723363] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/05:12:01:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 347 test script finished in 509.35s cleanup kill QemuMachine (pid 46) gitlab # [ 439.725889] sidekiq-cluster[3481]: {"severity":"INFO","time":"2026-09-05T12:01:05.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cec5bc1496a5d4965de152cc","created_at":"2026-09-05T12:01:02.145Z","trace_propagation_headers":{"sentry-trace":"5dfb2c94284443d2a18973d33953a2d8-f0138ef8ce9d484b","baggage":"sentry-trace_id=5dfb2c94284443d2a18973d33953a2d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0f4348d72a6947261c45e6c15585e71a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-05T12:01:02.221Z","job_size_bytes":2,"pid":3481,"sidekiq_tid":"m8rp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cec5bc1496a5d4965de152cc: done: 0.018425 sec","job_status":"done","queue_duration_s":2.998365,"scheduling_latency_s":2.998382,"redis_calls":10,"redis_duration_s":0.0013310000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"dqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) gitlab # b (finished: cleanup, in 1.63 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-05T11:59:20.866Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-05T12:00:55.603229Z', 'started_at': '2026-09-05T12:00:55.894894Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-05T12:00:55.859Z', 'started_at': None, 'finished_at': '2026-09-05T12:01:05.328Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '3dfbb554ce9ad1d04f68d6a5521ddd472709e6c1', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-05T12:01:05.501Z', 'started_at': None, 'finished_at': '2026-09-05T12:01:05.668Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]