tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.O9tXUo81uI', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 72.15 seconds) gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for unit setup-gitlab.service gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab: waiting for the VM to finish booting gitlab # Filesystem UUID: 481a3003-b9c3-4ca4-9683-a13805983e2e gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] KASLR enabled gitlab # [ 0.000000] random: crng init done gitlab # [ 0.000000] Machine model: linux,dummy-virt gitlab # [ 0.000000] efi: UEFI not found. gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] gitlab # [ 0.000000] Zone ranges: gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] gitlab # [ 0.000000] DMA32 empty gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.000000] Device empty gitlab # [ 0.000000] Movable zone start for each node gitlab # [ 0.000000] Early memory node ranges gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 gitlab # [ 0.000000] psci: probing for conduit method from DT. gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs gitlab # [ 0.000000] psci: Trusted OS migration not required gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface gitlab # [ 0.000000] CPU features: detected: Spectre-v4 gitlab # [ 0.000000] CPU features: detected: Spectre-BHB gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 gitlab # [ 0.000000] alternatives: applying boot alternatives gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/z5vyyrhw4aljdiw9yh434z0881rsjl2k-nixos-system-gitlab-test/init regInfo=/nix/store/69ap9qy873xk7hy3547i1dqz3x1cajw2-closure-info/registration console=ttyAMA0,115200n8 console=tty0 gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/69ap9qy873xk7hy3547i1dqz3x1cajw2-closure-info/registration", will be passed to user space. gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.000000] software IO TLB: area num 4. gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) gitlab # [ 0.000000] Fallback order for Node 0: 0 gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 gitlab # [ 0.000000] Policy zone: Normal gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.000000] allocated 12582912 bytes of page_ext gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. gitlab # [ 0.000000] rcu: RCU event tracing is enabled. gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 gitlab # [ 0.000000] GICv3: 256 SPIs implemented gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f40000 gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f50000 gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns gitlab # [ 0.000028] arm-pv: using stolen time PV gitlab # [ 0.000381] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.000579] Console: colour dummy device 80x25 gitlab # [ 0.000587] printk: legacy console [tty0] enabled gitlab # [ 0.000766] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) gitlab # [ 0.000772] pid_max: default: 32768 minimum: 301 gitlab # [ 0.000864] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.000991] landlock: Up and running. gitlab # [ 0.000994] Yama: becoming mindful. gitlab # [ 0.001390] LSM support for eBPF active gitlab # [ 0.001563] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.001666] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.002764] cacheinfo: Unable to detect cache hierarchy for CPU 0 gitlab # [ 0.003427] rcu: Hierarchical SRCU implementation. gitlab # [ 0.003431] rcu: Max phase no-delay instances is 1000. gitlab # [ 0.003579] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.004583] fsl-mc MSI: its@8080000 domain created gitlab # [ 0.004676] EFI services will not be available. gitlab # [ 0.004860] smp: Bringing up secondary CPUs ... gitlab # [ 0.005482] Detected PIPT I-cache on CPU1 gitlab # [ 0.005582] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 gitlab # [ 0.005704] GICv3: CPU1: using allocated LPI pending table @0x0000000100f60000 gitlab # [ 0.005827] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] gitlab # [ 0.006871] Detected PIPT I-cache on CPU2 gitlab # [ 0.006979] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 gitlab # [ 0.007102] GICv3: CPU2: using allocated LPI pending table @0x0000000100f70000 gitlab # [ 0.007226] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] gitlab # [ 0.008295] Detected PIPT I-cache on CPU3 gitlab # [ 0.008409] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 gitlab # [ 0.008531] GICv3: CPU3: using allocated LPI pending table @0x0000000100f80000 gitlab # [ 0.008655] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] gitlab # [ 0.009154] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.009182] SMP: Total of 4 processors activated. gitlab # [ 0.009184] CPU: All CPU(s) started at EL1 gitlab # [ 0.009192] CPU features: detected: Branch Target Identification gitlab # [ 0.009196] CPU features: detected: ARMv8.4 Translation Table Level gitlab # [ 0.009199] CPU features: detected: Instruction cache invalidation not required for I/D coherence gitlab # [ 0.009202] CPU features: detected: Data cache clean to the PoU not required for I/D coherence gitlab # [ 0.009206] CPU features: detected: Common not Private translations gitlab # [ 0.009209] CPU features: detected: CRC32 instructions gitlab # [ 0.009211] CPU features: detected: Data cache clean to Point of Deep Persistence gitlab # [ 0.009215] CPU features: detected: Data cache clean to Point of Persistence gitlab # [ 0.009218] CPU features: detected: Data independent timing control (DIT) gitlab # [ 0.009220] CPU features: detected: E0PD gitlab # [ 0.009223] CPU features: detected: Enhanced Counter Virtualization gitlab # [ 0.009226] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) gitlab # [ 0.009228] CPU features: detected: Enhanced Virtualization Traps gitlab # [ 0.009231] CPU features: detected: Fine Grained Traps gitlab # [ 0.009234] CPU features: detected: Generic authentication (architected QARMA5 algorithm) gitlab # [ 0.009238] CPU features: detected: RCpc load-acquire (LDAPR) gitlab # [ 0.009241] CPU features: detected: LSE atomic instructions gitlab # [ 0.009243] CPU features: detected: Privileged Access Never gitlab # [ 0.009246] CPU features: detected: PMUv3 gitlab # [ 0.009248] CPU features: detected: RAS Extension Support gitlab # [ 0.009251] CPU features: detected: RASv1p1 Extension Support gitlab # [ 0.009253] CPU features: detected: Random Number Generator gitlab # [ 0.009255] CPU features: detected: Speculation barrier (SB) gitlab # [ 0.009258] CPU features: detected: Stage-2 Force Write-Back gitlab # [ 0.009261] CPU features: detected: TLB range maintenance instructions gitlab # [ 0.009264] CPU features: detected: Speculative Store Bypassing Safe (SSBS) gitlab # [ 0.009389] alternatives: applying system-wide alternatives gitlab # [ 0.012202] CPU features: detected: BBM Level 2 without TLB conflict abort gitlab # [ 0.012409] Memory: 5963748K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) gitlab # [ 0.013855] devtmpfs: initialized gitlab # [ 0.017793] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.017857] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.018043] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL gitlab # [ 0.018048] 0 pages in range for non-PLT usage gitlab # [ 0.018049] 508288 pages in range for PLT usage gitlab # [ 0.018166] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.018904] DMI not present or invalid. gitlab # [ 0.021926] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.024565] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.024843] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.026560] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.026581] audit: initializing netlink subsys (disabled) gitlab # [ 0.026897] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.027381] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.027384] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.027388] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.027390] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.027393] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.027471] cpuidle: using governor ladder gitlab # [ 0.027494] cpuidle: using governor menu gitlab # [ 0.027690] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. gitlab # [ 0.027899] ASID allocator initialised with 65536 entries gitlab # [ 0.029009] Serial: AMBA PL011 UART driver gitlab # [ 0.034050] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 gitlab # [ 0.034378] printk: console [ttyAMA0] enabled gitlab # [ 0.044469] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.044491] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.044501] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages gitlab # [ 0.044510] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page gitlab # [ 0.044518] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.044526] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.044534] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages gitlab # [ 0.044542] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page gitlab # [ 0.058285] fbcon: Taking over console gitlab # [ 0.058296] ACPI: Interpreter disabled. gitlab # [ 0.059258] iommu: Default domain type: Translated gitlab # [ 0.059262] iommu: DMA domain TLB invalidation policy: strict mode gitlab # [ 0.068089] SCSI subsystem initialized gitlab # [ 0.068419] usbcore: registered new interface driver usbfs gitlab # [ 0.068444] usbcore: registered new interface driver hub gitlab # [ 0.068459] usbcore: registered new device driver usb gitlab # [ 0.068730] pps_core: LinuxPPS API ver. 1 registered gitlab # [ 0.068734] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti gitlab # [ 0.068743] PTP clock support registered gitlab # [ 0.068781] EDAC MC: Ver: 3.0.0 gitlab # [ 0.069183] scmi_core: SCMI protocol bus registered gitlab # [ 0.069664] FPGA manager framework gitlab # [ 0.070466] vgaarb: loaded gitlab # [ 0.072348] clocksource: Switched to clocksource arch_sys_counter gitlab # [ 0.072774] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.072796] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.073144] netfs: FS-Cache loaded gitlab # [ 0.073263] pnp: PnP ACPI: disabled gitlab # [ 0.089049] NET: Registered PF_INET protocol family gitlab # [ 0.091592] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.147299] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.147367] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.147425] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.147823] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.148693] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.148834] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.148904] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.149069] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.149328] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.149366] NET: Registered PF_XDP protocol family gitlab # [ 0.149386] PCI: CLS 0 bytes, default 64 gitlab # [ 0.149742] Trying to unpack rootfs image as initramfs... gitlab # [ 0.164474] kvm [1]: HYP mode not available gitlab # [ 0.240054] Initialise system trusted keyrings gitlab # [ 0.240459] workingset: timestamp_bits=42 max_order=21 bucket_order=0 gitlab # [ 0.241388] squashfs: version 4.0 (2009/01/31) Phillip Lougher gitlab # [ 0.241538] 9p: Installing v9fs 9p2000 file system support gitlab # [ 0.256760] Key type asymmetric registered gitlab # [ 0.256776] Asymmetric key parser 'x509' registered gitlab # [ 0.256853] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) gitlab # [ 0.257266] io scheduler mq-deadline registered gitlab # [ 0.257284] io scheduler kyber registered gitlab # [ 0.271258] pl061_gpio 9030000.pl061: PL061 GPIO chip registered gitlab # [ 0.276423] ledtrig-cpu: registered to indicate activity on CPUs gitlab # [ 0.277419] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: gitlab # [ 0.277454] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 gitlab # [ 0.277502] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 gitlab # [ 0.277522] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 gitlab # [ 0.277568] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits gitlab # [ 0.277638] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] gitlab # [ 0.277890] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 gitlab # [ 0.277910] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.277921] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] gitlab # [ 0.277930] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] gitlab # [ 0.277939] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] gitlab # [ 0.278086] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.279570] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.280168] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.280207] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.280289] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.280326] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.282110] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.282724] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.282759] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.282882] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.284771] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.285577] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.285613] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.285693] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.287675] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.288305] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.288340] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.288464] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.290105] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.290792] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.290839] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.290931] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.292678] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.293407] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.293443] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.293535] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.295271] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.295869] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.295905] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.295985] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.296022] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.297715] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint gitlab # [ 0.298329] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.298421] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.300136] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.300823] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.300919] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.302372] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint gitlab # [ 0.302966] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] gitlab # [ 0.303791] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.304536] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] gitlab # [ 0.304571] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.304652] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.306397] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.307029] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.307064] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.307143] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.309236] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned gitlab # [ 0.309255] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned gitlab # [ 0.309266] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned gitlab # [ 0.309396] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned gitlab # [ 0.309549] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned gitlab # [ 0.309683] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned gitlab # [ 0.309828] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned gitlab # [ 0.309965] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned gitlab # [ 0.310109] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned gitlab # [ 0.310255] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned gitlab # [ 0.310391] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned gitlab # [ 0.310539] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned gitlab # [ 0.310719] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned gitlab # [ 0.310851] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned gitlab # [ 0.310905] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned gitlab # [ 0.310959] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned gitlab # [ 0.311025] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned gitlab # [ 0.311079] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned gitlab # [ 0.311134] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned gitlab # [ 0.311208] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned gitlab # [ 0.311263] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned gitlab # [ 0.311318] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned gitlab # [ 0.311375] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned gitlab # [ 0.311430] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned gitlab # [ 0.311488] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned gitlab # [ 0.311541] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned gitlab # [ 0.311622] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned gitlab # [ 0.311676] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned gitlab # [ 0.311728] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned gitlab # [ 0.311782] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned gitlab # [ 0.311837] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned gitlab # [ 0.311890] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned gitlab # [ 0.311955] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned gitlab # [ 0.312010] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned gitlab # [ 0.312087] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] gitlab # [ 0.312102] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] gitlab # [ 0.312112] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] gitlab # [ 0.316755] pci 0000:00:0a.0: enabling device (0000 -> 0002) gitlab # [ 0.350957] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) gitlab # [ 0.356312] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) gitlab # [ 0.361540] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) gitlab # [ 0.368012] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) gitlab # [ 0.372999] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) gitlab # [ 0.379550] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) gitlab # [ 0.384651] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) gitlab # [ 0.389849] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) gitlab # [ 0.396334] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) gitlab # [ 0.401062] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) gitlab # [ 0.407924] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) gitlab # [ 0.420649] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.424837] msm_serial: driver initialized gitlab # [ 0.425245] SuperH (H)SCI(F) driver initialized gitlab # [ 0.425359] STM32 USART driver initialized gitlab # [ 0.467464] loop: module loaded gitlab # [ 0.467895] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 0.471687] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 0.475134] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 0.478630] virtio_blk virtio5: [vdb] 12237784 512-byte logical blocks (6.27 GB/5.83 GiB) gitlab # [ 0.482619] megasas: 07.734.00.00-rc1 gitlab # [ 0.483973] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] gitlab # [ 0.490336] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.490396] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.493598] Using buffer write method gitlab # [ 0.493652] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] gitlab # [ 0.497842] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.497870] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.500679] Using buffer write method gitlab # [ 0.500706] Concatenating MTD devices: gitlab # [ 0.500711] (0): "0.flash" gitlab # [ 0.500716] (1): "0.flash" gitlab # [ 0.500720] into device "0.flash" gitlab # [ 0.590592] Freeing initrd memory: 27184K gitlab # [ 0.610548] tun: Universal TUN/TAP device driver, 1.6 gitlab # [ 0.620061] thunder_xcv, ver 1.0 gitlab # [ 0.620171] thunder_bgx, ver 1.0 gitlab # [ 0.620255] nicpf, ver 1.0 gitlab # [ 0.622061] e1000: Intel(R) PRO/1000 Network Driver gitlab # [ 0.622074] e1000: Copyright (c) 1999-2006 Intel Corporation. gitlab # [ 0.622161] e1000e: Intel(R) PRO/1000 Network Driver gitlab # [ 0.622175] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. gitlab # [ 0.622246] igb: Intel(R) Gigabit Ethernet Network Driver gitlab # [ 0.622255] igb: Copyright (c) 2007-2014 Intel Corporation. gitlab # [ 0.622340] igbvf: Intel(R) Gigabit Virtual Function Network Driver gitlab # [ 0.622350] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. gitlab # [ 0.622738] sky2: driver version 1.30 gitlab # [ 0.627864] usbcore: registered new interface driver usb-storage gitlab # [ 0.628107] ehci-pci 0000:00:0a.0: EHCI Host Controller gitlab # [ 0.628156] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 gitlab # [ 0.628282] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.628349] usbserial: USB Serial support registered for generic gitlab # [ 0.628856] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 gitlab # [ 0.630014] hv_vmbus: registering driver hyperv_keyboard gitlab # [ 0.632466] rtc-pl031 9010000.pl031: registered as rtc0 gitlab # [ 0.632525] rtc-pl031 9010000.pl031: setting system clock to 2026-08-27T02:01:17 UTC (1787796077) gitlab # [ 0.633430] i2c_dev: i2c /dev entries driver gitlab # [ 0.640468] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 gitlab # [ 0.641115] sdhci: Secure Digital Host Controller Interface driver gitlab # [ 0.641126] sdhci: Copyright(c) Pierre Ossman gitlab # [ 0.641410] hub 1-0:1.0: USB hub found gitlab # [ 0.641495] hub 1-0:1.0: 6 ports detected gitlab # [ 0.641820] Synopsys Designware Multimedia Card Interface Driver gitlab # [ 0.642782] sdhci-pltfm: SDHCI platform and OF driver helper gitlab # [ 0.646657] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 0.647449] usbcore: registered new interface driver usbhid gitlab # [ 0.647457] usbhid: USB HID core driver gitlab # [ 0.652999] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available gitlab # [ 0.656612] drop_monitor: Initializing network drop monitor service gitlab # [ 0.656895] NET: Registered PF_INET6 protocol family gitlab # [ 0.658437] Segment Routing with IPv6 gitlab # [ 0.658462] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.658572] NET: Registered PF_PACKET protocol family gitlab # [ 0.658815] 9pnet: Installing 9P2000 support gitlab # [ 0.661988] Key type dns_resolver registered gitlab # [ 0.675823] registered taskstats version 1 gitlab # [ 0.676227] Loading compiled-in X.509 certificates gitlab # [ 0.693897] Demotion targets for Node 0: null gitlab # [ 0.694055] Key type .fscrypt registered gitlab # [ 0.694060] Key type fscrypt-provisioning registered gitlab # [ 0.694254] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.694272] ima: Allocated hash algorithm: sha1 gitlab # [ 0.694309] ima: No architecture policies found gitlab # [ 0.695298] input: gpio-keys as /devices/platform/gpio-keys/input/input0 gitlab # [ 0.717641] clk: Disabling unused clocks gitlab # [ 0.717670] PM: genpd: Disabling unused power domains gitlab # [ 0.722031] Freeing unused kernel memory: 4736K gitlab # [ 0.722253] Run /init as init process gitlab # [ 0.754554] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 0.888441] usb 1-1: new high-speed USB device number 2 using ehci-pci gitlab # [ 1.040652] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 gitlab # [ 1.088972] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.089038] systemd[1]: Detected virtualization qemu. gitlab # [ 1.089107] systemd[1]: Detected architecture arm64. gitlab # [ 1.089123] systemd[1]: Running in initrd. gitlab # [ 1.089915] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.090161] systemd[1]: Hostname set to . gitlab # [ 1.173086] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 gitlab # [ 1.292439] usb 1-2: new high-speed USB device number 3 using ehci-pci gitlab # [ 1.321705] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.410679] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.429155] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.429541] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.429595] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.429618] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.429664] systemd[1]: Reached target Path Units. gitlab # [ 1.429691] systemd[1]: Reached target Slice Units. gitlab # [ 1.429722] systemd[1]: Reached target Swaps. gitlab # [ 1.429754] systemd[1]: Reached target Timer Units. gitlab # [ 1.430005] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.430257] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.430493] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.430715] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.430851] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.430884] systemd[1]: Reached target Socket Units. gitlab # [ 1.433426] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.435981] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.436088] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 1.438617] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 1.448869] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 gitlab # [ 1.452760] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 gitlab # [ 1.466016] systemd[1]: Starting Journal Service... gitlab # [ 1.469826] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.470026] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.473520] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.482408] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.484918] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.485812] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.486915] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 1.491068] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.507477] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.508403] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.509028] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 gitlab # [ 1.509176] [drm] features: -virgl +edid -resource_blob -host_visible gitlab # [ 1.509179] [drm] features: -context_init gitlab # [ 1.509987] [drm] number of scanouts: 1 gitlab # [ 1.509999] [drm] number of cap sets: 0 gitlab # [ 1.519909] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 1.521855] systemd-journald[94]: Collecting audit messages is disabled. gitlab # [ 1.522466] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 1.532613] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic gitlab # [ 1.532644] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 gitlab # [ 1.536236] Console: switching to colour frame buffer device 160x50 gitlab # [ 1.542141] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device gitlab # [ 1.556133] systemd[1]: Finished Load Kernel Modules. gitlab # [ 1.558414] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.565563] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.566102] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.566631] systemd[1]: Reached target Local File Systems. gitlab # [ 1.572782] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.596924] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.592772] systemd-modules-load[95]: Using 4 probe threads gitlab # [ 1.599234] systemd[1]: Started Journal Service. gitlab # [ 1.598319] systemd-modules-load[95]: Module 'virtio_balloon' is built in gitlab # [ 1.600176] systemd-modules-load[95]: Module 'virtio_console' is built in gitlab # [ 1.602400] systemd-modules-load[95]: Module 'virtio_rng' is built in gitlab # [ 1.603947] systemd-modules-load[95]: Inserted module 'dm_mod' gitlab # [ 1.605501] systemd-modules-load[95]: Inserted module 'virtio_gpu' gitlab # [ 1.607439] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.627139] systemd-udevd[107]: Using default interface naming scheme 'v261'. gitlab # [ 1.641125] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.657407] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.695705] systemd[1]: Starting Virtual Console Setup... gitlab # [ 1.743307] systemd-vconsole-setup[133]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 1.747262] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.134316] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.136999] systemd[1]: Reached target System Initialization. gitlab # [ 2.139513] systemd[1]: Reached target Basic System. gitlab # [ 2.320768] (udev-worker)[149]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.351636] (udev-worker)[128]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.358768] (udev-worker)[128]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.362116] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.366118] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.368880] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.370463] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.415362] systemd-fsck[155]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.421895] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.472475] systemd[1]: Mounting /sysroot... gitlab # [ 2.517346] EXT4-fs (vda): mounted filesystem 481a3003-b9c3-4ca4-9683-a13805983e2e r/w with ordered data mode. Quota mode: none. gitlab # [ 2.515118] systemd[1]: Mounted /sysroot. gitlab # [ 2.516909] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.522783] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.525809] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.544406] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.551715] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 2.554211] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 2.567800] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 2.569025] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 2.569913] systemd[1]: Mounted /sysroot/run. gitlab # [ 2.583809] systemd-sysroot-fstab-check[169]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 2.590980] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.597958] systemd[1]: Reload requested from client PID 169 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 2.600337] systemd[1]: Reloading... gitlab # [ 2.846047] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 2.889517] systemd[1]: Reloading finished in 289 ms. gitlab # [ 2.936895] (udev-worker)[130]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory gitlab # [ 2.939797] (udev-worker)[130]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory gitlab # [ 2.945421] systemd-sysroot-fstab-check[169]: Requesting initrd-fs.target/start/replace... gitlab # [ 2.947985] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 2.949650] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 2.950728] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 2.951965] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 2.953156] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 2.954596] systemd-sysroot-fstab-check[169]: Requesting swap.target/start/replace... gitlab # [ 2.961363] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 2.962436] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 2.963393] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 2.969999] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.970899] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 2.971923] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 2.972801] systemd[1]: Stopping Virtual Console Setup... gitlab # [ 2.973581] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.992042] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 2.993592] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.003381] systemd-vconsole-setup[201]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 3.005388] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.478378] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.547824] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.550364] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.553179] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.555591] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.609557] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.613356] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.633853] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.636409] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.638921] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.693378] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.695926] systemd[1]: Stopped target Basic System. gitlab # [ 3.698140] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.700575] systemd[1]: Stopped target Path Units. gitlab # [ 3.702683] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.705984] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.709329] systemd[1]: Stopped target Slice Units. gitlab # [ 3.711480] systemd[1]: Stopped target Socket Units. gitlab # [ 3.714550] systemd[1]: Stopped target System Initialization. gitlab # [ 3.717493] systemd[1]: Stopped target Swaps. gitlab # [ 3.722300] systemd[1]: Stopped target Timer Units. gitlab # [ 3.724691] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.727225] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.730089] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.733456] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.735691] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 3.739014] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.743703] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.746566] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.748919] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.751794] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.753772] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.756635] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.759449] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.762343] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.764834] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.766819] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.769328] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.771846] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.773981] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.776561] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.778935] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.780638] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.782665] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.784634] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 3.786671] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 3.789556] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.791491] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.793180] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.794984] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.797197] systemd[1]: systemd-udevd.service: Consumed 2.383s CPU time over 2.193s wall clock time, 31.8M memory peak. gitlab # [ 3.799925] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.801973] systemd[1]: Closed udev Control Socket. gitlab # [ 3.803343] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.804877] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.806891] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.808615] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.810783] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.812757] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.814592] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 3.868271] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 3.871668] systemd[1]: Finished Cleanup udev Database. gitlab # [ 3.874104] systemd[1]: Reached target Switch Root. gitlab # [ 3.876347] systemd[1]: Starting NixOS Activation... gitlab # [ 4.072886] initrd-nixos-activation-start[229]: booting system configuration /nix/store/z5vyyrhw4aljdiw9yh434z0881rsjl2k-nixos-system-gitlab-test gitlab # [ 4.134385] initrd-nixos-activation-start[229]: running activation script... gitlab # [ 4.435203] initrd-nixos-activation-start[252]: setting up /etc... gitlab # [ 4.598681] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.603133] systemd[1]: Finished NixOS Activation. gitlab # [ 4.605363] systemd[1]: Starting Switch Root... gitlab # [ 4.658883] systemd[1]: Switching root. gitlab # [ 4.792957] systemd-journald[94]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.421089] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.421226] systemd[1]: Detected virtualization qemu. gitlab # [ 5.421302] systemd[1]: Detected architecture arm64. gitlab # [ 5.421474] systemd[1]: Detected first boot. gitlab # [ 5.422779] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.687313] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.816731] systemd[1]: Applying preset policy. gitlab # [ 6.035396] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.195067] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.195505] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.199932] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.207302] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.211123] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.215336] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.216864] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.218247] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.219405] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.220630] systemd[1]: Expecting device /dev/ttyAMA0... gitlab # [ 6.221742] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.222882] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.224010] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.225126] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.226248] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.227597] systemd[1]: Reached target Path Units. gitlab # [ 6.228717] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.229780] systemd[1]: Reached target Slice Units. gitlab # [ 6.230851] systemd[1]: Reached target Swaps. gitlab # [ 6.234845] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 6.239086] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.242311] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.246025] systemd[1]: Listening on Factory Reset Management. gitlab # [ 6.247367] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 6.256591] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.258550] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.262107] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.263448] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.264480] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.265308] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.269940] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.270925] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.271962] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 6.280463] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.287346] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.304218] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.317154] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.326211] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.336320] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 6.337679] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 6.348460] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.349605] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.350689] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.362199] systemd[1]: Starting Load Kernel Module fuse... gitlab # [ 6.363231] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.374100] systemd[1]: Starting Journal Service... gitlab # [ 6.379368] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.388536] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.395198] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.395266] fuse: init (API version 7.45) gitlab # [ 6.399820] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.418001] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.431062] systemd-journald[323]: Collecting audit messages is enabled. gitlab # [ 6.433021] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.433767] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.440603] EXT4-fs (vda): re-mounted 481a3003-b9c3-4ca4-9683-a13805983e2e. gitlab # [ 6.440838] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.442953] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.443733] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.444787] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.445590] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 6.446215] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 6.458968] systemd[1]: Queued start job for default target Multi-User System.[ 6.446753] systemd[1]: Mounted Kernel Configuration File System. gitlab # gitlab # [ 6.447394] systemd[1]: modprobe@fuse.service: Deactivated successfully. gitlab # [ 6.448101] systemd[1]: Finished Load Kernel Module fuse. gitlab # [ 6.455234] systemd[1]: Finished Load Kernel Modules. gitlab # [ 6.456155] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.457137] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.471896] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.476387] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.483709] systemd[1]: Starting Firewall... gitlab # [ 6.484188] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.487680] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.489223] systemd-modules-load[324]: Using 3 probe threads gitlab # [ 6.490182] systemd-modules-load[324]: Module 'atkbd' is built in gitlab # [ 6.491145] systemd-modules-load[324]: Module 'loop' is built in gitlab # [ 6.492152] systemd-modules-load[324]: Inserted module 'tls' gitlab # [ 6.500499] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.507231] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.505896] systemd-oomd[325]: No swap; memory pressure usage will be degraded gitlab # [ 6.513526] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.516543] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.519844] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.[ 6.518477] systemd[1]: Started Journal Service. gitlab # gitlab # [ 6.527400] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.532719] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.541244] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.546562] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.550601] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.556190] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.557960] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.587504] systemd-journald[323]: Received client request to flush runtime journal. gitlab # [ 6.604994] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.608544] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.609596] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.610888] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.658419] systemd-udevd[367]: Using default interface naming scheme 'v261'. gitlab # [ 6.689700] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 7.025836] systemd[1]: Finished Firewall. gitlab # [ 7.031480] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 7.112495] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 7.165601] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 7.202801] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.211123] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. gitlab # [ 7.240123] systemd[1]: Mounted /run/wrappers. gitlab # [ 7.241083] systemd[1]: Reached target Local File Systems. gitlab # [ 7.242066] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 7.245914] (udev-worker)[407]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.247681] systemd[1]: Starting register-nix-paths.service... gitlab # [ 7.252167] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 7.253355] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.258477] (udev-worker)[473]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 7.264517] (udev-worker)[473]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.267201] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 7.275107] systemd[1]: Starting Create System Files and Directories... gitlab # [ 7.295158] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 7.297431] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.299486] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 7.302326] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.305176] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.307129] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.313613] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.315888] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.318229] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.325045] systemd[1]: Finished Create System Files and Directories. gitlab # [ 7.330371] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 7.336713] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 7.371026] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.385276] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.391203] mousedev: PS/2 mouse device common for all mice gitlab # [ 7.389366] systemd[1]: Starting Update is Completed... gitlab # [ 7.417189] systemd[1]: Finished Update is Completed. gitlab # [ 7.460610] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.462045] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.463092] systemd[1]: suid-sgid-wrappers.service: Consumed 133ms CPU time over 210ms wall clock time, 6.4M memory peak, 1.9M read from disk. gitlab # [ 7.720494] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.722272] systemd[1]: Reached target System Initialization. gitlab # [ 7.724158] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.725657] systemd[1]: Started logrotate.timer. gitlab # [ 7.726414] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.727631] systemd[1]: Reached target Timer Units. gitlab # [ 7.728530] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.729942] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.741620] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.744252] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.745670] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.746584] systemd[1]: Reached target Socket Units. gitlab # [ 7.747676] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.787880] dbus-broker-launch[567]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.789503] dbus-broker-launch[567]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.790617] dbus-broker-launch[567]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.794665] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.814675] dbus-broker-launch[567]: Ready gitlab # [ 7.815406] systemd[1]: Reached target Basic System. gitlab # [ 7.816183] systemd[1]: Started backdoor.service. gitlab # [ 7.824583] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.848532] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.854705] systemd[1]: Starting Logrotate configuration check... gitlab # connecting to host... gitlab # [ 7.871077] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 gitlab # [ 7.870842] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab: connected to guest root shell gitlab # [ 7.883390] systemd[1]: Starting Post-Boot Actions... gitlab: (connecting took 8.33 seconds) gitlab # [ 7.885367] logrotate[576]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab: (finished: waiting for the VM to finish booting, in 8.33 seconds) gitlab # [ 7.887317] logrotate[576]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf gitlab # [ 7.888661] logrotate[576]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.889622] logrotate[576]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.890628] logrotate[576]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.891612] logrotate[576]: Allocating hash table for state file, size 64 entries gitlab # [ 7.892956] logrotate[576]: Handling 4 logs gitlab # [ 7.893504] logrotate[576]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.895658] logrotate[576]: considering log /var/log/btmp gitlab # [ 7.896447] logrotate[576]: Creating new state gitlab # [ 7.897201] logrotate[576]: Now: 2026-08-27 02:01 gitlab # [ 7.897856] logrotate[576]: Last rotated at 2026-08-27 02:00 gitlab # [ 7.898628] logrotate[576]: log does not need rotating (log has already been rotated) gitlab # [ 7.902236] logrotate[576]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.904446] logrotate[576]: considering log /var/log/wtmp gitlab # [ 7.905170] logrotate[576]: Creating new state gitlab # [ 7.905916] logrotate[576]: Now: 2026-08-27 02:01 gitlab # [ 7.906650] logrotate[576]: Last rotated at 2026-08-27 02:00 gitlab # [ 7.907463] logrotate[576]: log does not need rotating (log has already been rotated) gitlab # [ 7.908548] logrotate[576]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.910291] logrotate[576]: switching euid from 0 to 165 and egid from 0 to 165 (pid 576) gitlab # [ 7.911342] logrotate[576]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.912279] logrotate[576]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.913330] logrotate[576]: Creating new state gitlab # [ 7.913955] logrotate[576]: switching euid from 165 to 0 and egid from 165 to 0 (pid 576) gitlab # [ 7.915168] logrotate[576]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.916919] logrotate[576]: switching euid from 0 to 60 and egid from 0 to 60 (pid 576) gitlab # [ 7.917969] logrotate[576]: considering log /var/log/nginx/*.log gitlab # [ 7.918762] logrotate[576]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.919728] logrotate[576]: Creating new state gitlab # [ 7.920366] logrotate[576]: not running postrotate script, since no logs were rotated gitlab # [ 7.921385] logrotate[576]: switching euid from 60 to 0 and egid from 60 to 0 (pid 576) gitlab # [ 7.922475] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.923772] nsncd[577]: Aug 27 02:01:24.764 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.926582] systemd[1]: Starting resolvconf update... gitlab # [ 7.927382] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.946160] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.947664] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.957616] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.977244] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.978252] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.994533] systemd[1]: Starting User Login Management... gitlab # [ 7.999095] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.012112] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 8.013271] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 8.016406] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 8.017569] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 8.020126] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 8.021291] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 8.022191] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 8.023222] systemd[1]: nscd.service: Consumed 25ms CPU time over 144ms wall clock time, 7.6M memory peak, 5.8M read from disk. gitlab # [ 8.028321] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.044385] systemd-logind[646]: New seat seat0. gitlab # [ 8.048774] systemd-logind[646]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 8.050077] systemd-logind[646]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) gitlab # [ 8.054042] systemd-logind[646]: Watching system buttons on /dev/input/event0 (gpio-keys) gitlab # [ 8.056212] systemd[1]: Started User Login Management. gitlab # [ 8.056962] systemd[1]: Starting linger-users.service... gitlab # [ 8.057735] systemd[1]: Finished resolvconf update. gitlab # [ 8.058454] systemd[1]: Reached target Preparation for Network. gitlab # [ 8.064506] systemd[1]: Starting DHCP Client... gitlab # [ 8.067091] systemd[1]: Starting Address configuration of eth1... gitlab # [ 8.070498] nsncd[669]: Aug 27 02:01:24.939 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.074452] systemd-vconsole-setup[652]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 8.077724] systemd[1]: Starting Extra networking commands.... gitlab # [ 8.078738] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.079680] systemd[1]: Finished Virtual Console Setup. gitlab # [ 8.083642] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 8.084672] systemd[1]: Finished linger-users.service. gitlab # [ 8.086188] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.087134] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.103155] network-addresses-eth1-start[687]: adding address 192.168.1.1/24... done gitlab # [ 8.108444] network-addresses-eth1-start[687]: adding address 2001:db8:1::1/64... done gitlab # [ 8.115660] systemd[1]: Finished Address configuration of eth1. gitlab # [ 8.156163] systemd[1]: Finished Extra networking commands.. gitlab # [ 8.157918] systemd[1]: Reached target Network. gitlab # [ 8.160619] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 8.165206] systemd[1]: Starting Nginx Web Server... gitlab # [ 8.170203] systemd[1]: Starting PostgreSQL Server... gitlab # [ 8.173773] dhcpcd[715]: dhcpcd-10.3.2 starting gitlab # [ 8.174533] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 8.176826] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.181675] dhcpcd[768]: dev: loaded udev gitlab # [ 8.195874] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 8.196298] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.201884] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.221623] systemd[1]: Finished Permit User Sessions. gitlab # [ 8.234980] systemd[1]: Started Getty on tty1.[ 8.239426] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # gitlab # [ 8.237710] systemd[1]: Reached target Login Prompts. gitlab # [ 8.243831] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 8.244341] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 8.245971] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 8.246326] cfg80211: failed to load regulatory.db gitlab # [ 8.282563] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 8.281185] dhcpcd[768]: eth0: waiting for carrier gitlab # [ 8.282807] dhcpcd[768]: eth0: carrier acquired gitlab # [ 8.286370] dhcpcd[768]: DUID 00:01:00:01:32:22:56:f5:52:54:00:12:34:56 gitlab # [ 8.287462] dhcpcd[768]: eth0: IAID 00:12:34:56 gitlab # [ 8.288650] dhcpcd[768]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 8.331920] nginx-pre-start[798]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok gitlab # [ 8.333977] nginx-pre-start[798]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful gitlab # [ 8.337542] systemd[1]: Started Nginx Web Server. gitlab # [ 8.371644] redis-server[794]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 8.376067] redis-server[794]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 8.377829] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 8.378859] redis-server[794]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=794, just started gitlab # [ 8.380793] redis-server[794]: Configuration loaded gitlab # [ 8.382113] redis[794]: monotonic clock: ARM CNTVCT @ 1000 ticks/us gitlab # [ 8.383520] redis[794]: Running mode=standalone, port=0. gitlab # [ 8.384614] redis[794]: Server initialized gitlab # [ 8.385389] redis[794]: Ready to accept connections unix gitlab # [ 8.386609] redis[794]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 8.403139] postgresql-pre-start[803]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.405085] postgresql-pre-start[803]: This user must also own the server process. gitlab # [ 8.407525] postgresql-pre-start[803]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.408987] postgresql-pre-start[803]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.412171] postgresql-pre-start[803]: The default text search configuration will be set to "english". gitlab # [ 8.413562] postgresql-pre-start[803]: Data page checksums are enabled. gitlab # [ 8.414526] postgresql-pre-start[803]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 8.415881] postgresql-pre-start[803]: creating subdirectories ... ok gitlab # [ 8.417318] postgresql-pre-start[803]: selecting dynamic shared memory implementation ... posix gitlab # [ 8.454696] postgresql-pre-start[803]: selecting default "max_connections" ... 100 gitlab # [ 8.505740] postgresql-pre-start[803]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.822167] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.947075] postgresql-pre-start[803]: selecting default time zone ... UTC gitlab # [ 8.950219] postgresql-pre-start[803]: creating configuration files ... ok gitlab # [ 9.193225] postgresql-pre-start[803]: running bootstrap script ... ok gitlab # [ 9.228711] gitlab-workhorse[824]: time="2026-08-27T02:01:26Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 9.230584] gitlab-workhorse[824]: time="2026-08-27T02:01:26Z" level=info msg="Using redis/go-redis" gitlab # [ 9.231904] gitlab-workhorse[824]: time="2026-08-27T02:01:26Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 9.876378] dhcpcd[768]: eth0: soliciting a DHCP lease gitlab # [ 9.893179] dhcpcd[768]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.920574] dhcpcd[768]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.926314] postgresql-pre-start[803]: performing post-bootstrap initialization ... ok gitlab # [ 9.949341] dhcpcd[768]: eth0: soliciting an IPv6 router gitlab # [ 9.959014] dhcpcd[768]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.967943] dhcpcd[768]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.974313] dhcpcd[768]: eth0: adding route to fec0::/64 gitlab # [ 9.977745] dhcpcd[768]: eth0: adding default route via fe80::2 gitlab # [ 10.210306] postgresql-pre-start[803]: syncing data to disk ... ok gitlab # [ 10.211520] postgresql-pre-start[803]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 10.213388] postgresql-pre-start[803]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 10.216071] postgresql-pre-start[803]: Success. You can now start the database server using: gitlab # [ 10.217484] postgresql-pre-start[803]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 10.426672] postgres[857]: [857] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 10.432452] postgres[857]: [857] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 10.435808] postgres[857]: [857] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 10.440262] postgres[857]: [857] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 10.446217] postgres[867]: [867] LOG: database system was shut down at 2026-08-27 02:01:26 GMT gitlab # [ 10.455778] postgres[857]: [857] LOG: database system is ready to accept connections gitlab # [ 10.462667] systemd[1]: Started PostgreSQL Server. gitlab # [ 10.467998] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.858582] postgresql-setup-start[878]: CREATE DATABASE gitlab # [ 10.863087] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[883]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 10.866987] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[883]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 10.907943] postgresql-setup-start[897]: CREATE ROLE gitlab # [ 10.922699] postgresql-setup-start[901]: ALTER DATABASE gitlab # [ 10.927156] systemd[1]: Finished gitlab-config.service. gitlab # [ 10.935137] systemd[1]: Started gitaly.service. gitlab # [ 10.983326] postgresql-setup-start[908]: CREATE ROLE gitlab # [ 10.988400] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 10.990838] systemd[1]: Reached target PostgreSQL. gitlab # [ 10.995483] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 11.001104] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 11.228801] gitlab-postgresql-start[919]: CREATE DATABASE gitlab # [ 11.283569] gitlab-postgresql-start[939]: CREATE EXTENSION gitlab # [ 11.372927] gitlab-postgresql-start[941]: CREATE EXTENSION gitlab # [ 11.377737] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 11.576112] gitaly[906]: time="2026-08-27T02:01:28.446Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=906 gitlab # [ 11.586734] gitaly[906]: time="2026-08-27T02:01:28.447Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=906 gitlab # [ 11.591677] gitaly[906]: time="2026-08-27T02:01:28.447Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=906 gitlab # [ 11.597578] gitaly[906]: time="2026-08-27T02:01:28.447Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=906 gitlab # [ 11.603083] gitaly[906]: time="2026-08-27T02:01:28.447Z" level=info msg="Starting Gitaly" pid=906 version= gitlab # [ 11.779089] gitaly[906]: time="2026-08-27T02:01:28.650Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=202 pid=906 gitlab # [ 11.781150] gitaly[906]: time="2026-08-27T02:01:28.650Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=906 gitlab # [ 11.782767] gitaly[906]: time="2026-08-27T02:01:28.651Z" level=info msg="finished initializing command factory" duration_ms=0 pid=906 gitlab # [ 11.784578] gitaly[906]: time="2026-08-27T02:01:28.651Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-906/git-exec-2622890959.d/git pid=906 gitlab # [ 11.786667] gitaly[906]: time="2026-08-27T02:01:28.656Z" level=info msg="finished detecting git version" duration_ms=5 pid=906 gitlab # [ 11.788316] gitaly[906]: time="2026-08-27T02:01:28.656Z" level=info msg="using Git version" pid=906 version=2.53.g gitlab # [ 11.789752] gitaly[906]: time="2026-08-27T02:01:28.656Z" level=info msg="clearing disk cache object folder" pid=906 storage=default gitlab # [ 11.791374] gitaly[906]: time="2026-08-27T02:01:28.657Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache196759572 pid=906 storage=default gitlab # [ 11.793932] gitaly[906]: time="2026-08-27T02:01:28.657Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache196759572 pid=906 storage=default gitlab # [ 11.796930] gitaly[906]: time="2026-08-27T02:01:28.657Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=906 gitlab # [ 11.800794] gitaly[906]: time="2026-08-27T02:01:28.657Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=906 gitlab # [ 11.803582] gitaly[906]: time="2026-08-27T02:01:28.657Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache196759572 clear_duration_ms=0 pid=906 storage=default gitlab # [ 11.806565] gitaly[906]: time="2026-08-27T02:01:28.671Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=906 gitlab # [ 11.808772] gitaly[906]: time="2026-08-27T02:01:28.671Z" level=info msg="counting repositories" pid=906 gitlab # [ 11.810020] gitaly[906]: time="2026-08-27T02:01:28.671Z" level=info msg="starting to count repositories" pid=906 storage_path=/var/gitlab/state/repositories gitlab # [ 11.811825] gitaly[906]: time="2026-08-27T02:01:28.671Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=906 schema=unix gitlab # [ 11.813857] gitaly[906]: time="2026-08-27T02:01:28.671Z" level=info msg="finished tempdir cleaner walk" pid=906 storage=default time_ms=0 gitlab # [ 11.815434] gitaly[906]: time="2026-08-27T02:01:28.671Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=906 storage_path=/var/gitlab/state/repositories gitlab # [ 11.817603] gitaly[906]: time="2026-08-27T02:01:28.671Z" level=info msg="completed counting all repositories" counting_duration=0 pid=906 gitlab # [ 11.819197] gitaly[906]: time="2026-08-27T02:01:28.671Z" level=info msg="listening at address" address=/run/gitaly/gitaly-906/sock.d/intern pid=906 schema=unix gitlab # [ 11.821100] gitaly[906]: time="2026-08-27T02:01:28.673Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=906 gitlab # [ 11.822653] gitaly[906]: time="2026-08-27T02:01:28.673Z" level=info msg="Started Gitaly" duration_ms=226 pid=906 gitlab # [ 11.823964] gitaly[906]: time="2026-08-27T02:01:28.673Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=906 gitlab # [ 11.825663] gitaly[906]: time="2026-08-27T02:01:28.673Z" level=info msg="maintenance: daily scheduled" pid=906 scheduled="2026-08-27 12:00:00 +0000 UTC" gitlab # [ 12.438514] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.449158] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.459231] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 14.855373] dhcpcd[768]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.858534] dhcpcd[768]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.868221] dhcpcd[768]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.981838] systemd[1]: Started DHCP Client. gitlab # [ 14.984224] systemd[1]: Reached target Network is Online. gitlab # [ 15.330383] gitaly[906]: time="2026-08-27T02:01:32.201Z" level=info msg="License database preloaded" duration_ms=3753 pid=906 gitlab # [ 58.329271] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: W, [2026-08-27T02:02:15.199086 #917] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 59.313054] gitlab-workhorse[824]: time="2026-08-27T02:02:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 62.963771] postgres[1458]: [1458] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 62.965378] postgres[1458]: [1458] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 62.966625] postgres[1458]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 62.967928] postgres[1458]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 62.969168] postgres[1458]: attidentity AS identity, gitlab # [ 62.969969] postgres[1458]: attgenerated as attgenerated gitlab # [ 62.970809] postgres[1458]: FROM pg_attribute a gitlab # [ 62.971492] postgres[1458]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 62.972832] postgres[1458]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 62.973752] postgres[1458]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 62.975149] postgres[1458]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 62.976110] postgres[1458]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 62.977013] postgres[1458]: ORDER BY a.attnum gitlab # [ 62.977663] postgres[1458]: gitlab # [ 68.504537] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1509]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 68.505022] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1509]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 69.314544] gitlab-workhorse[824]: time="2026-08-27T02:02:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.322769] gitlab-workhorse[824]: time="2026-08-27T02:02:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 89.330324] gitlab-workhorse[824]: time="2026-08-27T02:02:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 96.292318] postgres[1621]: [1621] ERROR: relation "table_without_models" does not exist gitlab # [ 96.296438] postgres[1621]: [1621] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 99.338428] gitlab-workhorse[824]: time="2026-08-27T02:02:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.348581] gitlab-workhorse[824]: time="2026-08-27T02:03:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 119.358292] gitlab-workhorse[824]: time="2026-08-27T02:03:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 129.253321] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 129.264074] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 129.266325] systemd[1]: Starting gitlab.service... gitlab # [ 129.362772] gitlab-workhorse[824]: time="2026-08-27T02:03:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 129.854373] bundle[2023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 129.860794] bundle[2023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 129.866881] bundle[2023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 134.832120] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.667Z","pid":2023,"message":"* Enabling systemd notification integration"} gitlab # [ 134.832622] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.702Z","pid":2023,"message":"Puma starting in cluster mode..."} gitlab # [ 134.832774] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.702Z","pid":2023,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 134.832914] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.702Z","pid":2023,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 134.833018] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.702Z","pid":2023,"message":"* Min threads: 0"} gitlab # [ 134.833131] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.702Z","pid":2023,"message":"* Max threads: 4"} gitlab # [ 134.833232] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.702Z","pid":2023,"message":"* Environment: production"} gitlab # [ 134.833331] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.702Z","pid":2023,"message":"* Master PID: 2023"} gitlab # [ 134.833442] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.702Z","pid":2023,"message":"* Workers: 2"} gitlab # [ 134.833557] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.703Z","pid":2023,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 134.833675] bundle[2023]: {"timestamp":"2026-08-27T02:03:31.703Z","pid":2023,"message":"* Preloading application"} gitlab # [ 134.944454] sidekiq-cluster[2022]: {"severity":"INFO","time":"2026-08-27T02:03:31.775Z","message":"Starting cluster with 1 processes"} gitlab # [ 139.366937] gitlab-workhorse[824]: time="2026-08-27T02:03:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 149.367993] gitlab-workhorse[824]: time="2026-08-27T02:03:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 159.374486] gitlab-workhorse[824]: time="2026-08-27T02:03:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 169.385226] gitlab-workhorse[824]: time="2026-08-27T02:04:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 175.374331] bundle[2023]: W, [2026-08-27T02:04:12.244769 #2023] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 179.386640] gitlab-workhorse[824]: time="2026-08-27T02:04:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 180.312808] sidekiq-cluster[2073]: 2026-08-27T02:04:17.183Z pid=2073 tid=xpt INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2073, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>427991040} gitlab # [ 180.620933] sidekiq-cluster[2073]: W, [2026-08-27T02:04:17.492390 #2073] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 180.780392] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:17.651Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 184.239007] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.109Z","message":"GitLab reliable fetch activated!"} gitlab # [ 184.239506] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.109Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 184.271518] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.143Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 184.278751] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.146Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 184.286014] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.148Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 184.292801] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.151Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 184.300310] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.154Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 184.308326] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.157Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 184.315276] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.160Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 184.321753] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.162Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 184.327417] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.164Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 184.332737] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.166Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 184.337923] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.168Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 184.342800] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.170Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 184.347057] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.172Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 184.351365] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.175Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 184.355467] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.177Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 184.359359] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.179Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 184.362849] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.183Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 184.366095] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.186Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 184.369476] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.188Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 184.372554] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.191Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 184.375780] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.194Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 184.379333] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.196Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 184.382300] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.198Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 184.385165] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.201Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 184.387911] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.203Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 184.390676] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.206Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 184.393176] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.209Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 184.395591] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.211Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 184.398228] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.214Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 184.400755] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.217Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 184.402979] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.220Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 184.405588] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.223Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 184.407758] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.226Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 184.410265] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.229Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 184.412570] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.231Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 184.414804] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.234Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 184.417081] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.236Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 184.419146] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.239Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 184.421377] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.242Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 184.423433] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.244Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 184.425813] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.246Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 184.428017] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.248Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 184.430229] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.250Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 184.432368] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.253Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 184.434619] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.256Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 184.437023] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.258Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 184.439220] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.260Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 184.441573] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.262Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 184.443712] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.265Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 184.446047] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.268Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 184.448214] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.270Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 184.450335] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.273Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 184.452513] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.288Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 184.454915] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.291Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 184.457237] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.294Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 184.459438] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.297Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 184.461803] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.300Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 184.464146] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.302Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 184.466410] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.305Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 184.468773] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.307Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 184.470897] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.309Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 184.473147] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.311Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 184.475406] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.313Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 184.477866] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.316Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 184.523992] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.394Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 184.527526] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.399Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 184.530859] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.401Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 184.534258] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.403Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 184.537782] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.407Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 184.541321] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.409Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 184.544672] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.411Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 184.547959] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.415Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 184.551485] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.417Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 184.555209] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.420Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 184.558851] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.422Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 184.562476] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.425Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 184.565887] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.428Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 184.569623] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.430Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 184.572784] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.433Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 184.575729] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.435Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 184.579161] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.438Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 184.582501] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.440Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 184.585552] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.444Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 184.588154] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.446Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 184.590801] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.450Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 184.593376] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.452Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 184.595962] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.455Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 184.598739] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.458Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 184.601405] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.460Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 184.603889] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.463Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 184.606600] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.465Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 184.609057] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.467Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 184.611427] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.469Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 184.614088] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.472Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 184.616148] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.475Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 184.618558] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.478Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 184.620964] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.480Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 184.623271] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.483Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 184.626065] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.497Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 184.629508] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.501Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 184.631747] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.503Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 184.635810] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.507Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 184.638928] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.510Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 184.641723] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.513Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 184.645533] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.517Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 184.648383] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.519Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 184.650919] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.522Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 184.654096] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.525Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 184.657023] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:04:21.528Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 189.394306] gitlab-workhorse[824]: time="2026-08-27T02:04:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 199.402243] gitlab-workhorse[824]: time="2026-08-27T02:04:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 209.410303] gitlab-workhorse[824]: time="2026-08-27T02:04:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 219.420459] gitlab-workhorse[824]: time="2026-08-27T02:04:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 225.158988] postgres[2828]: [2828] ERROR: relation "table_without_models" does not exist gitlab # [ 225.162799] postgres[2828]: [2828] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 228.576404] postgres[2865]: [2865] ERROR: relation "table_without_models" does not exist gitlab # [ 228.579877] postgres[2865]: [2865] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 229.430396] gitlab-workhorse[824]: time="2026-08-27T02:05:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 230.123627] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:06.994Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 230.129128] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:07.000Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 230.131349] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:07.000Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 230.136712] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:07.000Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 230.143019] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:07.005Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 232.870491] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:09.740Z","message":"Cleaning working queues"} gitlab # [ 232.874371] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:09.745Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"91bab86dc8c704bfcb47a321","created_at":"2026-08-27T02:03:25.539Z","trace_propagation_headers":{"sentry-trace":"c18ab5cc4b194896b499c11fba93b078-88b89b2f057547db","baggage":"sentry-trace_id=c18ab5cc4b194896b499c11fba93b078,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"cd5dca86f2ae028aa7c2d2b3e57af435","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-27T02:03:25.552Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-91bab86dc8c704bfcb47a321: start","job_status":"start","queue_duration_s":104.192934,"scheduling_latency_s":104.192998} gitlab # [ 233.308221] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:10.178Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"91bab86dc8c704bfcb47a321","created_at":"2026-08-27T02:03:25.539Z","trace_propagation_headers":{"sentry-trace":"c18ab5cc4b194896b499c11fba93b078-88b89b2f057547db","baggage":"sentry-trace_id=c18ab5cc4b194896b499c11fba93b078,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"cd5dca86f2ae028aa7c2d2b3e57af435","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-27T02:03:25.552Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-91bab86dc8c704bfcb47a321: done: 0.433493 sec","job_status":"done","queue_duration_s":104.192934,"scheduling_latency_s":104.192998,"redis_calls":29,"redis_duration_s":0.012828,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007564,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003192,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002072,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.20445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.20570387999999998,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.433493,"completed_at":"2026-08-27T02:05:10.178Z","load_balancing_strategy":"primary","db_duration_s":0.109145,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 236.919962] bundle[2023]: {"timestamp":"2026-08-27T02:05:13.790Z","pid":2023,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 236.925034] bundle[2023]: {"timestamp":"2026-08-27T02:05:13.790Z","pid":2023,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 236.925520] bundle[2023]: {"timestamp":"2026-08-27T02:05:13.790Z","pid":2023,"message":"! #\u003cThread:0x0000fcfa1357f980 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 236.932321] bundle[2023]: {"timestamp":"2026-08-27T02:05:13.790Z","pid":2023,"message":"! #\u003cThread:0x0000fcfa12ece7f8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 236.937321] bundle[2023]: {"timestamp":"2026-08-27T02:05:13.790Z","pid":2023,"message":"Use Ctrl-C to stop"} gitlab # [ 236.938576] bundle[2023]: {"timestamp":"2026-08-27T02:05:13.791Z","pid":2023,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 237.681533] bundle[2023]: {"timestamp":"2026-08-27T02:05:14.552Z","pid":2023,"message":"- Worker 0 (PID: 2993) booted in 0.7s, phase: 0"} gitlab # [ 237.743479] bundle[2023]: {"timestamp":"2026-08-27T02:05:14.614Z","pid":2023,"message":"- Worker 1 (PID: 2997) booted in 0.73s, phase: 0"} gitlab # [ 237.750080] systemd[1]: Started gitlab.service. gitlab # [ 237.752386] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 237.758167] systemd[1]: Starting setup-gitlab.service... gitlab # [ 237.807557] setup-gitlab-start[3030]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 241.630612] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:05:18 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 3766 gitlab # [ 241.640000] setup-gitlab-start[3030]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 241.643408] setup-gitlab-start[3030]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 241.672431] runuser[3060]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 242.253535] setup-gitlab-start[3061]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 242.260613] setup-gitlab-start[3061]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 242.267669] setup-gitlab-start[3061]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 243.140752] setup-gitlab-start[3061]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 243.141135] setup-gitlab-start[3061]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 243.143712] setup-gitlab-start[3061]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 247.067961] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:23.937Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3a9b1a5532791256470f02c1","created_at":"2026-08-27T02:05:23.837Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a88a8e2cc5b9797aa4dea7a43a26c4c6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:23.933Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3a9b1a5532791256470f02c1: start","job_status":"start","queue_duration_s":0.004148,"scheduling_latency_s":0.004238} gitlab # [ 247.339541] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:24.209Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3a9b1a5532791256470f02c1","created_at":"2026-08-27T02:05:23.837Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a88a8e2cc5b9797aa4dea7a43a26c4c6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:23.933Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3a9b1a5532791256470f02c1: done: 0.271883 sec","job_status":"done","queue_duration_s":0.004148,"scheduling_latency_s":0.004238,"redis_calls":16,"redis_duration_s":0.043251000000000005,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.028688,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004917,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00208716,"duration_s":0.271883,"completed_at":"2026-08-27T02:05:24.209Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.068139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.434767] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:24.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e31faf5ba9a2b48d89133c7b","created_at":"2026-08-27T02:05:23.952Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:23.972Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e31faf5ba9a2b48d89133c7b: start","job_status":"start","queue_duration_s":0.245621,"scheduling_latency_s":0.245683} gitlab # [ 248.209848] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.078Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e31faf5ba9a2b48d89133c7b","created_at":"2026-08-27T02:05:23.952Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:23.972Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e31faf5ba9a2b48d89133c7b: done: 0.861069 sec","job_status":"done","queue_duration_s":0.245621,"scheduling_latency_s":0.245683,"redis_calls":71,"redis_duration_s":0.31935,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.018485,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.002111,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.296163,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.353741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00346038,"duration_s":0.861069,"completed_at":"2026-08-27T02:05:25.078Z","load_balancing_strategy":"primary","db_duration_s":0.029418,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.305594] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.084Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8b0d840e685f3661675311b4","created_at":"2026-08-27T02:05:24.012Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff38ed10144527916807c0256404f0d6","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.029Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8b0d840e685f3661675311b4: start","job_status":"start","queue_duration_s":1.054933,"scheduling_latency_s":1.054996} gitlab # [ 248.309512] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.179Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8b0d840e685f3661675311b4","created_at":"2026-08-27T02:05:24.012Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff38ed10144527916807c0256404f0d6","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.029Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8b0d840e685f3661675311b4: done: 0.095761 sec","job_status":"done","queue_duration_s":1.054933,"scheduling_latency_s":1.054996,"redis_calls":12,"redis_duration_s":0.005298000000000001,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000879,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002379,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00165482,"exclusive_lock_hold_duration_s":0.010841,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.095761,"completed_at":"2026-08-27T02:05:25.179Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.397297] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"71dbff797f4544341253107d","created_at":"2026-08-27T02:05:24.049Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7eadc6d581a3a1b90954d109abe7eafe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-27T02:05:24.076Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-71dbff797f4544341253107d: start","job_status":"start","queue_duration_s":1.106757,"scheduling_latency_s":1.106808} gitlab # [ 248.411536] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.226Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 248.415613] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.267Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"71dbff797f4544341253107d","created_at":"2026-08-27T02:05:24.049Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7eadc6d581a3a1b90954d109abe7eafe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-27T02:05:24.076Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-71dbff797f4544341253107d: done: 0.083983 sec","job_status":"done","queue_duration_s":1.106757,"scheduling_latency_s":1.106808,"redis_calls":10,"redis_duration_s":0.003751,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00078,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00214,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005803,"duration_s":0.083983,"completed_at":"2026-08-27T02:05:25.267Z","load_balancing_strategy":"primary","db_duration_s":0.00315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.459321] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"719c75d537f5c4dba610832b","created_at":"2026-08-27T02:05:24.085Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8172bb34da68dd865ed26c7533fb08c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.113Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-719c75d537f5c4dba610832b: start","job_status":"start","queue_duration_s":1.165922,"scheduling_latency_s":1.165995} gitlab # [ 248.740409] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"719c75d537f5c4dba610832b","created_at":"2026-08-27T02:05:24.085Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8172bb34da68dd865ed26c7533fb08c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.113Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-719c75d537f5c4dba610832b: done: 0.330477 sec","job_status":"done","queue_duration_s":1.165922,"scheduling_latency_s":1.165995,"redis_calls":14,"redis_duration_s":0.01251,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005423,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004458,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.027,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00355388,"exclusive_lock_hold_duration_s":0.21526086,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.330477,"completed_at":"2026-08-27T02:05:25.609Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.066078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.839840] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"94d3a2ae5f28d68b55bab6c9","created_at":"2026-08-27T02:05:24.141Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25f93cc7aac72e1d05e708ae84e649e0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.171Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-94d3a2ae5f28d68b55bab6c9: start","job_status":"start","queue_duration_s":1.443643,"scheduling_latency_s":1.443696} gitlab # [ 248.857619] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"94d3a2ae5f28d68b55bab6c9","created_at":"2026-08-27T02:05:24.141Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25f93cc7aac72e1d05e708ae84e649e0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.171Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-94d3a2ae5f28d68b55bab6c9: done: 0.06095 sec","job_status":"done","queue_duration_s":1.443643,"scheduling_latency_s":1.443696,"redis_calls":10,"redis_duration_s":0.004232,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000789,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002584,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000859,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064296,"duration_s":0.06095,"completed_at":"2026-08-27T02:05:25.675Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.902826] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d39daa965f1ee9b1e82306ad","created_at":"2026-08-27T02:05:24.198Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bea6fca5d2f5e91fafbd765cbc9260a5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-27T02:05:24.280Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d39daa965f1ee9b1e82306ad: start","job_status":"start","queue_duration_s":1.398541,"scheduling_latency_s":1.398583} gitlab # [ 248.917873] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d39daa965f1ee9b1e82306ad","created_at":"2026-08-27T02:05:24.198Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bea6fca5d2f5e91fafbd765cbc9260a5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-27T02:05:24.280Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d39daa965f1ee9b1e82306ad: done: 0.067664 sec","job_status":"done","queue_duration_s":1.398541,"scheduling_latency_s":1.398583,"redis_calls":10,"redis_duration_s":0.00396,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00083,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002342,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058368,"duration_s":0.067664,"completed_at":"2026-08-27T02:05:25.746Z","load_balancing_strategy":"primary","db_duration_s":0.008778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.963742] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.749Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b0b594cd77209ad8a99c930a","created_at":"2026-08-27T02:05:24.295Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de2537a680fee86b0152405c39e25e49","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.320Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b0b594cd77209ad8a99c930a: start","job_status":"start","queue_duration_s":1.429426,"scheduling_latency_s":1.429469} gitlab # [ 248.980216] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.802Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b0b594cd77209ad8a99c930a","created_at":"2026-08-27T02:05:24.295Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de2537a680fee86b0152405c39e25e49","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.320Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b0b594cd77209ad8a99c930a: done: 0.052613 sec","job_status":"done","queue_duration_s":1.429426,"scheduling_latency_s":1.429469,"redis_calls":12,"redis_duration_s":0.00435,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000727,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001982,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001641,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00125138,"exclusive_lock_hold_duration_s":0.01549374,"duration_s":0.052613,"completed_at":"2026-08-27T02:05:25.802Z","load_balancing_strategy":"primary","db_duration_s":0.007744,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.025147] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"43928ca3cfd0f8b818abc497","created_at":"2026-08-27T02:05:24.339Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.382Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-43928ca3cfd0f8b818abc497: start","job_status":"start","queue_duration_s":1.422705,"scheduling_latency_s":1.422742} gitlab # [ 249.039366] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.878Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"43928ca3cfd0f8b818abc497","created_at":"2026-08-27T02:05:24.339Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.382Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-43928ca3cfd0f8b818abc497: done: 0.073657 sec","job_status":"done","queue_duration_s":1.422705,"scheduling_latency_s":1.422742,"redis_calls":19,"redis_duration_s":0.011053,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002818,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000435,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006647,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001153,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00086112,"duration_s":0.073657,"completed_at":"2026-08-27T02:05:25.878Z","load_balancing_strategy":"primary","db_duration_s":0.004413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.084783] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7e8c9a6611590f692ae2cb33","created_at":"2026-08-27T02:05:24.451Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fefca1cffbb46c80b4b9a5297ad36dc0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.488Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7e8c9a6611590f692ae2cb33: start","job_status":"start","queue_duration_s":1.395231,"scheduling_latency_s":1.395272} gitlab # [ 249.100168] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7e8c9a6611590f692ae2cb33","created_at":"2026-08-27T02:05:24.451Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fefca1cffbb46c80b4b9a5297ad36dc0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.488Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7e8c9a6611590f692ae2cb33: done: 0.055307 sec","job_status":"done","queue_duration_s":1.395231,"scheduling_latency_s":1.395272,"redis_calls":13,"redis_duration_s":0.005058,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000815,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002466,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001777,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01616542,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.055307,"completed_at":"2026-08-27T02:05:25.939Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005995,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 249.147738] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"8958faf96f2e584a64beba4c","created_at":"2026-08-27T02:05:24.515Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69a95d34cc862e9959d0d82dbdaab47e","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.546Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-8958faf96f2e584a64beba4c: start","job_status":"start","queue_duration_s":1.3945,"scheduling_latency_s":1.394538} gitlab # [ 249.162000] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.991Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"8958faf96f2e584a64beba4c","created_at":"2026-08-27T02:05:24.515Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69a95d34cc862e9959d0d82dbdaab47e","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.546Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-8958faf96f2e584a64beba4c: done: 0.049694 sec","job_status":"done","queue_duration_s":1.3945,"scheduling_latency_s":1.394538,"redis_calls":10,"redis_duration_s":0.003741,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000795,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002185,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00067386,"duration_s":0.049694,"completed_at":"2026-08-27T02:05:25.991Z","load_balancing_strategy":"primary","db_duration_s":0.009136,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.205629] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:25.995Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ba67b037137726dd145cda14","created_at":"2026-08-27T02:05:24.581Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d845dc71909e34a00f589eabaad62f77","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.607Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ba67b037137726dd145cda14: start","job_status":"start","queue_duration_s":1.387834,"scheduling_latency_s":1.38787} gitlab # [ 249.611400] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ba67b037137726dd145cda14","created_at":"2026-08-27T02:05:24.581Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d845dc71909e34a00f589eabaad62f77","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.607Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ba67b037137726dd145cda14: done: 0.486162 sec","job_status":"done","queue_duration_s":1.387834,"scheduling_latency_s":1.38787,"redis_calls":565,"redis_duration_s":0.130474,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000755,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001643,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.128076,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.406737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037812,"duration_s":0.486162,"completed_at":"2026-08-27T02:05:26.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.696391] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.483Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3db194c00fbbea12f84575ec","created_at":"2026-08-27T02:05:24.681Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ec7e2ebd320ada67bd1dfb1e93e04fc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.712Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3db194c00fbbea12f84575ec: start","job_status":"start","queue_duration_s":1.771194,"scheduling_latency_s":1.771226} gitlab # [ 249.713882] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.571Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3db194c00fbbea12f84575ec","created_at":"2026-08-27T02:05:24.681Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ec7e2ebd320ada67bd1dfb1e93e04fc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.712Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3db194c00fbbea12f84575ec: done: 0.088099 sec","job_status":"done","queue_duration_s":1.771194,"scheduling_latency_s":1.771226,"redis_calls":12,"redis_duration_s":0.00526,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000864,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002016,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00238,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00176622,"exclusive_lock_hold_duration_s":0.02743412,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.088099,"completed_at":"2026-08-27T02:05:26.571Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.764791] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c22aacf6dcd8c19646102cc2","created_at":"2026-08-27T02:05:24.739Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5156f0e958b175d3b3366728fb1f550c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.773Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c22aacf6dcd8c19646102cc2: start","job_status":"start","queue_duration_s":1.802054,"scheduling_latency_s":1.802115} gitlab # [ 249.781548] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.603Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 249.785634] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c22aacf6dcd8c19646102cc2","created_at":"2026-08-27T02:05:24.739Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5156f0e958b175d3b3366728fb1f550c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.773Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c22aacf6dcd8c19646102cc2: done: 0.042578 sec","job_status":"done","queue_duration_s":1.802054,"scheduling_latency_s":1.802115,"redis_calls":10,"redis_duration_s":0.003567,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000635,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002227,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005096,"duration_s":0.042578,"completed_at":"2026-08-27T02:05:26.617Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.829505] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.620Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1d4afe887940df4e66da48f7","created_at":"2026-08-27T02:05:24.815Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1219c03f4ff72abb17abb81a26493669","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.848Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1d4afe887940df4e66da48f7: start","job_status":"start","queue_duration_s":1.772199,"scheduling_latency_s":1.772244} gitlab # [ 249.843911] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1d4afe887940df4e66da48f7","created_at":"2026-08-27T02:05:24.815Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1219c03f4ff72abb17abb81a26493669","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.848Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1d4afe887940df4e66da48f7: done: 0.081397 sec","job_status":"done","queue_duration_s":1.772199,"scheduling_latency_s":1.772244,"redis_calls":10,"redis_duration_s":0.0032930000000000004,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000577,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00065992,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.081397,"completed_at":"2026-08-27T02:05:26.701Z","load_balancing_strategy":"primary","db_duration_s":0.031839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.888149] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.704Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a0ecf5594d155ff4ea54264e","created_at":"2026-08-27T02:05:24.888Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6279d2c20819601641f8f0fa4ad7aca9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-27T02:05:24.909Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a0ecf5594d155ff4ea54264e: start","job_status":"start","queue_duration_s":1.794897,"scheduling_latency_s":1.794983} gitlab # [ 249.901976] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.729Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 249.905443] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a0ecf5594d155ff4ea54264e","created_at":"2026-08-27T02:05:24.888Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6279d2c20819601641f8f0fa4ad7aca9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-27T02:05:24.909Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a0ecf5594d155ff4ea54264e: done: 0.037458 sec","job_status":"done","queue_duration_s":1.794897,"scheduling_latency_s":1.794983,"redis_calls":10,"redis_duration_s":0.00286,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000622,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001601,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043454,"duration_s":0.037458,"completed_at":"2026-08-27T02:05:26.742Z","load_balancing_strategy":"primary","db_duration_s":0.000972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.948726] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:26.744Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ff7921da997ad258fa2cf0e4","created_at":"2026-08-27T02:05:24.942Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17ba48684ddc24fe7a9b08b447d8700f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.968Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ff7921da997ad258fa2cf0e4: start","job_status":"start","queue_duration_s":1.77616,"scheduling_latency_s":1.776202} gitlab # [ 250.239993] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.110Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ff7921da997ad258fa2cf0e4","created_at":"2026-08-27T02:05:24.942Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17ba48684ddc24fe7a9b08b447d8700f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.968Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ff7921da997ad258fa2cf0e4: done: 0.366212 sec","job_status":"done","queue_duration_s":1.77616,"scheduling_latency_s":1.776202,"redis_calls":12,"redis_duration_s":0.003477,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001461,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.058,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.289324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00097492,"exclusive_lock_hold_duration_s":0.33307857999999996,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.366212,"completed_at":"2026-08-27T02:05:27.110Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.087857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.338108] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"163737f5f08e6410868eb4f9","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-163737f5f08e6410868eb4f9: start","job_status":"start","queue_duration_s":2.131963,"scheduling_latency_s":2.131993} gitlab # [ 250.359154] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"163737f5f08e6410868eb4f9","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-163737f5f08e6410868eb4f9: done: 0.029249 sec","job_status":"done","queue_duration_s":2.131963,"scheduling_latency_s":2.131993,"redis_calls":16,"redis_duration_s":0.00317,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001849,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036094,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029249,"completed_at":"2026-08-27T02:05:27.142Z","load_balancing_strategy":"primary","db_duration_s":0.000711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.417614] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"609e3270f943de0c3d718854","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-609e3270f943de0c3d718854: start","job_status":"start","queue_duration_s":2.163013,"scheduling_latency_s":2.163045} gitlab # [ 250.430801] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"609e3270f943de0c3d718854","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-609e3270f943de0c3d718854: done: 0.020187 sec","job_status":"done","queue_duration_s":2.163013,"scheduling_latency_s":2.163045,"redis_calls":13,"redis_duration_s":0.002143,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001325,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020187,"completed_at":"2026-08-27T02:05:27.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.471237] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf2257338d44bc9bc0f07c37","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf2257338d44bc9bc0f07c37: start","job_status":"start","queue_duration_s":2.184711,"scheduling_latency_s":2.18474} gitlab # [ 250.484089] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf2257338d44bc9bc0f07c37","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf2257338d44bc9bc0f07c37: done: 0.020497 sec","job_status":"done","queue_duration_s":2.184711,"scheduling_latency_s":2.18474,"redis_calls":13,"redis_duration_s":0.002409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020497,"completed_at":"2026-08-27T02:05:27.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.523443] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"923c56b6e7bc806581ac4c8e","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-923c56b6e7bc806581ac4c8e: start","job_status":"start","queue_duration_s":2.206833,"scheduling_latency_s":2.206868} gitlab # [ 250.536373] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"923c56b6e7bc806581ac4c8e","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-923c56b6e7bc806581ac4c8e: done: 0.170697 sec","job_status":"done","queue_duration_s":2.206833,"scheduling_latency_s":2.206868,"redis_calls":13,"redis_duration_s":0.002255,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.162305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.170697,"completed_at":"2026-08-27T02:05:27.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.577407] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c8a416cd9eaeb695b9b852b","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c8a416cd9eaeb695b9b852b: start","job_status":"start","queue_duration_s":2.379356,"scheduling_latency_s":2.379389} gitlab # [ 250.590122] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c8a416cd9eaeb695b9b852b","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c8a416cd9eaeb695b9b852b: done: 0.022127 sec","job_status":"done","queue_duration_s":2.379356,"scheduling_latency_s":2.379389,"redis_calls":13,"redis_duration_s":0.002379,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022127,"completed_at":"2026-08-27T02:05:27.382Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.631574] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9da03c73b57fad611a0d97b5","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9da03c73b57fad611a0d97b5: start","job_status":"start","queue_duration_s":2.403098,"scheduling_latency_s":2.403129} gitlab # [ 250.644656] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9da03c73b57fad611a0d97b5","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9da03c73b57fad611a0d97b5: done: 0.019911 sec","job_status":"done","queue_duration_s":2.403098,"scheduling_latency_s":2.403129,"redis_calls":13,"redis_duration_s":0.002432,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001327,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019911,"completed_at":"2026-08-27T02:05:27.403Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.686424] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66393d75c9bde83256f7d81b","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66393d75c9bde83256f7d81b: start","job_status":"start","queue_duration_s":2.424724,"scheduling_latency_s":2.424754} gitlab # [ 250.699810] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66393d75c9bde83256f7d81b","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66393d75c9bde83256f7d81b: done: 0.020729 sec","job_status":"done","queue_duration_s":2.424724,"scheduling_latency_s":2.424754,"redis_calls":13,"redis_duration_s":0.00232,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001431,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020729,"completed_at":"2026-08-27T02:05:27.426Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.741631] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f0f97ae90172153b78f0360","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f0f97ae90172153b78f0360: start","job_status":"start","queue_duration_s":2.447701,"scheduling_latency_s":2.447736} gitlab # [ 250.756064] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f0f97ae90172153b78f0360","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f0f97ae90172153b78f0360: done: 0.02116 sec","job_status":"done","queue_duration_s":2.447701,"scheduling_latency_s":2.447736,"redis_calls":13,"redis_duration_s":0.002231,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02116,"completed_at":"2026-08-27T02:05:27.449Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.796083] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41f2cd2a4791306220b036af","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41f2cd2a4791306220b036af: start","job_status":"start","queue_duration_s":2.469994,"scheduling_latency_s":2.470024} gitlab # [ 250.808681] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41f2cd2a4791306220b036af","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41f2cd2a4791306220b036af: done: 0.020505 sec","job_status":"done","queue_duration_s":2.469994,"scheduling_latency_s":2.470024,"redis_calls":13,"redis_duration_s":0.002312,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001357,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020505,"completed_at":"2026-08-27T02:05:27.471Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.848917] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b7c5477c47e773605dfd108","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b7c5477c47e773605dfd108: start","job_status":"start","queue_duration_s":2.492176,"scheduling_latency_s":2.492207} gitlab # [ 250.861610] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b7c5477c47e773605dfd108","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b7c5477c47e773605dfd108: done: 0.020746 sec","job_status":"done","queue_duration_s":2.492176,"scheduling_latency_s":2.492207,"redis_calls":13,"redis_duration_s":0.002293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001445,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020746,"completed_at":"2026-08-27T02:05:27.493Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.901801] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f9bf2d26eb23ae1b7211595","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f9bf2d26eb23ae1b7211595: start","job_status":"start","queue_duration_s":2.514533,"scheduling_latency_s":2.514561} gitlab # [ 250.914539] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f9bf2d26eb23ae1b7211595","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f9bf2d26eb23ae1b7211595: done: 0.02117 sec","job_status":"done","queue_duration_s":2.514533,"scheduling_latency_s":2.514561,"redis_calls":13,"redis_duration_s":0.002249,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02117,"completed_at":"2026-08-27T02:05:27.516Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.954910] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cf4f724911f8fd7c7b622ec","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cf4f724911f8fd7c7b622ec: start","job_status":"start","queue_duration_s":2.537363,"scheduling_latency_s":2.537393} gitlab # [ 250.968698] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cf4f724911f8fd7c7b622ec","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cf4f724911f8fd7c7b622ec: done: 0.020854 sec","job_status":"done","queue_duration_s":2.537363,"scheduling_latency_s":2.537393,"redis_calls":13,"redis_duration_s":0.002217,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020854,"completed_at":"2026-08-27T02:05:27.539Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.008898] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b75e90e088664b6c50ca53b","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b75e90e088664b6c50ca53b: start","job_status":"start","queue_duration_s":2.560783,"scheduling_latency_s":2.560813} gitlab # [ 251.021666] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b75e90e088664b6c50ca53b","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b75e90e088664b6c50ca53b: done: 0.021137 sec","job_status":"done","queue_duration_s":2.560783,"scheduling_latency_s":2.560813,"redis_calls":13,"redis_duration_s":0.0023409999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001473,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021137,"completed_at":"2026-08-27T02:05:27.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.062277] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2dd497445526efb3a2ba9bc","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2dd497445526efb3a2ba9bc: start","job_status":"start","queue_duration_s":2.583576,"scheduling_latency_s":2.583608} gitlab # [ 251.074914] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2dd497445526efb3a2ba9bc","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2dd497445526efb3a2ba9bc: done: 0.02123 sec","job_status":"done","queue_duration_s":2.583576,"scheduling_latency_s":2.583608,"redis_calls":13,"redis_duration_s":0.002307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02123,"completed_at":"2026-08-27T02:05:27.585Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.115495] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fdda60e6a88368b40110144","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fdda60e6a88368b40110144: start","job_status":"start","queue_duration_s":2.606429,"scheduling_latency_s":2.606459} gitlab # [ 251.128707] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fdda60e6a88368b40110144","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fdda60e6a88368b40110144: done: 0.021513 sec","job_status":"done","queue_duration_s":2.606429,"scheduling_latency_s":2.606459,"redis_calls":13,"redis_duration_s":0.002358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001457,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021513,"completed_at":"2026-08-27T02:05:27.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.170365] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d274268934af8df04f1e7515","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d274268934af8df04f1e7515: start","job_status":"start","queue_duration_s":2.630581,"scheduling_latency_s":2.630615} gitlab # [ 251.183166] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d274268934af8df04f1e7515","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d274268934af8df04f1e7515: done: 0.021229 sec","job_status":"done","queue_duration_s":2.630581,"scheduling_latency_s":2.630615,"redis_calls":13,"redis_duration_s":0.0022110000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021229,"completed_at":"2026-08-27T02:05:27.632Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.223880] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87712e006c9ffc25335a9b27","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87712e006c9ffc25335a9b27: start","job_status":"start","queue_duration_s":2.653423,"scheduling_latency_s":2.653449} gitlab # [ 251.236912] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87712e006c9ffc25335a9b27","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87712e006c9ffc25335a9b27: done: 0.017122 sec","job_status":"done","queue_duration_s":2.653423,"scheduling_latency_s":2.653449,"redis_calls":13,"redis_duration_s":0.001639,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000609,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017122,"completed_at":"2026-08-27T02:05:27.651Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.276871] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06532dc54943eb2090f91170","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06532dc54943eb2090f91170: start","job_status":"start","queue_duration_s":2.671691,"scheduling_latency_s":2.671708} gitlab # [ 251.289582] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06532dc54943eb2090f91170","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06532dc54943eb2090f91170: done: 0.01579 sec","job_status":"done","queue_duration_s":2.671691,"scheduling_latency_s":2.671708,"redis_calls":13,"redis_duration_s":0.001539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01579,"completed_at":"2026-08-27T02:05:27.668Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.330651] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a56dfad9fae1d2798ca22418","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a56dfad9fae1d2798ca22418: start","job_status":"start","queue_duration_s":2.688567,"scheduling_latency_s":2.688584} gitlab # [ 251.344005] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a56dfad9fae1d2798ca22418","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a56dfad9fae1d2798ca22418: done: 0.015529 sec","job_status":"done","queue_duration_s":2.688567,"scheduling_latency_s":2.688584,"redis_calls":13,"redis_duration_s":0.001522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015529,"completed_at":"2026-08-27T02:05:27.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.386041] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46c6059b2af94a6507c5e390","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46c6059b2af94a6507c5e390: start","job_status":"start","queue_duration_s":2.705186,"scheduling_latency_s":2.705204} gitlab # [ 251.398844] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46c6059b2af94a6507c5e390","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46c6059b2af94a6507c5e390: done: 0.016738 sec","job_status":"done","queue_duration_s":2.705186,"scheduling_latency_s":2.705204,"redis_calls":13,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016738,"completed_at":"2026-08-27T02:05:27.702Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.439429] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc21d182a0e2c5d662385312","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc21d182a0e2c5d662385312: start","job_status":"start","queue_duration_s":2.723348,"scheduling_latency_s":2.723368} gitlab # [ 251.452382] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc21d182a0e2c5d662385312","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc21d182a0e2c5d662385312: done: 0.015744 sec","job_status":"done","queue_duration_s":2.723348,"scheduling_latency_s":2.723368,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015744,"completed_at":"2026-08-27T02:05:27.719Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.492978] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"791cc6b8c405cbe09e4c9766","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-791cc6b8c405cbe09e4c9766: start","job_status":"start","queue_duration_s":2.740193,"scheduling_latency_s":2.740211} gitlab # [ 251.505788] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"791cc6b8c405cbe09e4c9766","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-791cc6b8c405cbe09e4c9766: done: 0.015652 sec","job_status":"done","queue_duration_s":2.740193,"scheduling_latency_s":2.740211,"redis_calls":13,"redis_duration_s":0.001515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015652,"completed_at":"2026-08-27T02:05:27.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.546324] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e89ec37bd64529e44141d56","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e89ec37bd64529e44141d56: start","job_status":"start","queue_duration_s":2.756957,"scheduling_latency_s":2.756974} gitlab # [ 251.559235] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e89ec37bd64529e44141d56","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e89ec37bd64529e44141d56: done: 0.015836 sec","job_status":"done","queue_duration_s":2.756957,"scheduling_latency_s":2.756974,"redis_calls":13,"redis_duration_s":0.0015370000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015836,"completed_at":"2026-08-27T02:05:27.753Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.600785] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5508b1588c76ed91fee37fe8","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5508b1588c76ed91fee37fe8: start","job_status":"start","queue_duration_s":2.773872,"scheduling_latency_s":2.773889} gitlab # [ 251.613594] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5508b1588c76ed91fee37fe8","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5508b1588c76ed91fee37fe8: done: 0.019408 sec","job_status":"done","queue_duration_s":2.773872,"scheduling_latency_s":2.773889,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019408,"completed_at":"2026-08-27T02:05:27.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.654235] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3b7a15bb717aa62b5ed8a31","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3b7a15bb717aa62b5ed8a31: start","job_status":"start","queue_duration_s":2.794372,"scheduling_latency_s":2.79439} gitlab # [ 251.667035] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3b7a15bb717aa62b5ed8a31","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3b7a15bb717aa62b5ed8a31: done: 0.015458 sec","job_status":"done","queue_duration_s":2.794372,"scheduling_latency_s":2.79439,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015458,"completed_at":"2026-08-27T02:05:27.790Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.706949] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c18b0e413eed77c409fc4d5","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c18b0e413eed77c409fc4d5: start","job_status":"start","queue_duration_s":2.81093,"scheduling_latency_s":2.810948} gitlab # [ 251.719782] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c18b0e413eed77c409fc4d5","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c18b0e413eed77c409fc4d5: done: 0.016142 sec","job_status":"done","queue_duration_s":2.81093,"scheduling_latency_s":2.810948,"redis_calls":13,"redis_duration_s":0.0017959999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016142,"completed_at":"2026-08-27T02:05:27.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.760435] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56ccb164475accd5c6f6260f","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56ccb164475accd5c6f6260f: start","job_status":"start","queue_duration_s":2.828284,"scheduling_latency_s":2.828303} gitlab # [ 251.774009] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56ccb164475accd5c6f6260f","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56ccb164475accd5c6f6260f: done: 0.015564 sec","job_status":"done","queue_duration_s":2.828284,"scheduling_latency_s":2.828303,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015564,"completed_at":"2026-08-27T02:05:27.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.814343] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd34c50dba8a288edc598377","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd34c50dba8a288edc598377: start","job_status":"start","queue_duration_s":2.845845,"scheduling_latency_s":2.845864} gitlab # [ 251.827245] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd34c50dba8a288edc598377","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd34c50dba8a288edc598377: done: 0.015751 sec","job_status":"done","queue_duration_s":2.845845,"scheduling_latency_s":2.845864,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015751,"completed_at":"2026-08-27T02:05:27.842Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.867701] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b34e1437f01c2c6506da567f","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b34e1437f01c2c6506da567f: start","job_status":"start","queue_duration_s":2.862705,"scheduling_latency_s":2.862723} gitlab # [ 251.880733] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b34e1437f01c2c6506da567f","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b34e1437f01c2c6506da567f: done: 0.016224 sec","job_status":"done","queue_duration_s":2.862705,"scheduling_latency_s":2.862723,"redis_calls":13,"redis_duration_s":0.001705,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016224,"completed_at":"2026-08-27T02:05:27.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.921261] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb3161c17254a5f06a5de139","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb3161c17254a5f06a5de139: start","job_status":"start","queue_duration_s":2.88012,"scheduling_latency_s":2.880139} gitlab # [ 251.934497] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb3161c17254a5f06a5de139","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb3161c17254a5f06a5de139: done: 0.016549 sec","job_status":"done","queue_duration_s":2.88012,"scheduling_latency_s":2.880139,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016549,"completed_at":"2026-08-27T02:05:27.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.975857] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb45e73fa568590c608e91b3","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb45e73fa568590c608e91b3: start","job_status":"start","queue_duration_s":2.897768,"scheduling_latency_s":2.897786} gitlab # [ 251.989742] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb45e73fa568590c608e91b3","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb45e73fa568590c608e91b3: done: 0.016479 sec","job_status":"done","queue_duration_s":2.897768,"scheduling_latency_s":2.897786,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016479,"completed_at":"2026-08-27T02:05:27.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.029949] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9fc10b255ef837140356df8","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9fc10b255ef837140356df8: start","job_status":"start","queue_duration_s":2.916933,"scheduling_latency_s":2.916953} gitlab # [ 252.042690] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9fc10b255ef837140356df8","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9fc10b255ef837140356df8: done: 0.017359 sec","job_status":"done","queue_duration_s":2.916933,"scheduling_latency_s":2.916953,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017359,"completed_at":"2026-08-27T02:05:27.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.083221] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0aae81dea82f78cbc3de55b0","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0aae81dea82f78cbc3de55b0: start","job_status":"start","queue_duration_s":2.935512,"scheduling_latency_s":2.93553} gitlab # [ 252.096118] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0aae81dea82f78cbc3de55b0","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0aae81dea82f78cbc3de55b0: done: 0.015932 sec","job_status":"done","queue_duration_s":2.935512,"scheduling_latency_s":2.93553,"redis_calls":13,"redis_duration_s":0.00155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015932,"completed_at":"2026-08-27T02:05:27.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.136476] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0c6853304a4fbf1b55722cd","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0c6853304a4fbf1b55722cd: start","job_status":"start","queue_duration_s":2.952584,"scheduling_latency_s":2.952603} gitlab # [ 252.150459] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0c6853304a4fbf1b55722cd","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0c6853304a4fbf1b55722cd: done: 0.01555 sec","job_status":"done","queue_duration_s":2.952584,"scheduling_latency_s":2.952603,"redis_calls":13,"redis_duration_s":0.001563,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01555,"completed_at":"2026-08-27T02:05:27.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.192263] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e164b1dfc88c005328dc642","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e164b1dfc88c005328dc642: start","job_status":"start","queue_duration_s":2.969315,"scheduling_latency_s":2.969332} gitlab # [ 252.204932] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e164b1dfc88c005328dc642","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e164b1dfc88c005328dc642: done: 0.017428 sec","job_status":"done","queue_duration_s":2.969315,"scheduling_latency_s":2.969332,"redis_calls":13,"redis_duration_s":0.001715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017428,"completed_at":"2026-08-27T02:05:27.967Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.244934] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83c4634d9e60579e3deb604e","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83c4634d9e60579e3deb604e: start","job_status":"start","queue_duration_s":2.987939,"scheduling_latency_s":2.987957} gitlab # [ 252.257602] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83c4634d9e60579e3deb604e","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83c4634d9e60579e3deb604e: done: 0.016463 sec","job_status":"done","queue_duration_s":2.987939,"scheduling_latency_s":2.987957,"redis_calls":13,"redis_duration_s":0.001706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016463,"completed_at":"2026-08-27T02:05:27.985Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.297608] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:27.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb1d81792e5eadd4ef4cb214","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb1d81792e5eadd4ef4cb214: start","job_status":"start","queue_duration_s":3.005544,"scheduling_latency_s":3.005563} gitlab # [ 252.310065] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb1d81792e5eadd4ef4cb214","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb1d81792e5eadd4ef4cb214: done: 0.015715 sec","job_status":"done","queue_duration_s":3.005544,"scheduling_latency_s":3.005563,"redis_calls":13,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015715,"completed_at":"2026-08-27T02:05:28.002Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.350316] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5611af1a445fe767599e02d3","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5611af1a445fe767599e02d3: start","job_status":"start","queue_duration_s":3.022494,"scheduling_latency_s":3.022512} gitlab # [ 252.363133] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5611af1a445fe767599e02d3","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5611af1a445fe767599e02d3: done: 0.016077 sec","job_status":"done","queue_duration_s":3.022494,"scheduling_latency_s":3.022512,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016077,"completed_at":"2026-08-27T02:05:28.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.404571] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dc924eb2b37404ebf2eb885","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dc924eb2b37404ebf2eb885: start","job_status":"start","queue_duration_s":3.04011,"scheduling_latency_s":3.040131} gitlab # [ 252.417353] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dc924eb2b37404ebf2eb885","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dc924eb2b37404ebf2eb885: done: 0.016106 sec","job_status":"done","queue_duration_s":3.04011,"scheduling_latency_s":3.040131,"redis_calls":13,"redis_duration_s":0.0016840000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016106,"completed_at":"2026-08-27T02:05:28.037Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.457651] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fd86126e79e11a25524e063","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fd86126e79e11a25524e063: start","job_status":"start","queue_duration_s":3.057502,"scheduling_latency_s":3.057522} gitlab # [ 252.470437] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fd86126e79e11a25524e063","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fd86126e79e11a25524e063: done: 0.016097 sec","job_status":"done","queue_duration_s":3.057502,"scheduling_latency_s":3.057522,"redis_calls":13,"redis_duration_s":0.001866,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016097,"completed_at":"2026-08-27T02:05:28.054Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.510593] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55644e4e5d586c359b84a3d8","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55644e4e5d586c359b84a3d8: start","job_status":"start","queue_duration_s":3.074891,"scheduling_latency_s":3.074911} gitlab # [ 252.523421] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55644e4e5d586c359b84a3d8","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55644e4e5d586c359b84a3d8: done: 0.01659 sec","job_status":"done","queue_duration_s":3.074891,"scheduling_latency_s":3.074911,"redis_calls":13,"redis_duration_s":0.001709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01659,"completed_at":"2026-08-27T02:05:28.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.563760] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fe584ddba09efc27d95cb65","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fe584ddba09efc27d95cb65: start","job_status":"start","queue_duration_s":3.092696,"scheduling_latency_s":3.092716} gitlab # [ 252.576743] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fe584ddba09efc27d95cb65","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fe584ddba09efc27d95cb65: done: 0.016138 sec","job_status":"done","queue_duration_s":3.092696,"scheduling_latency_s":3.092716,"redis_calls":13,"redis_duration_s":0.001703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016138,"completed_at":"2026-08-27T02:05:28.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.617769] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"248b64f600ba7ad93843d829","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-248b64f600ba7ad93843d829: start","job_status":"start","queue_duration_s":3.110354,"scheduling_latency_s":3.110373} gitlab # [ 252.630510] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"248b64f600ba7ad93843d829","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-248b64f600ba7ad93843d829: done: 0.016663 sec","job_status":"done","queue_duration_s":3.110354,"scheduling_latency_s":3.110373,"redis_calls":13,"redis_duration_s":0.001692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016663,"completed_at":"2026-08-27T02:05:28.107Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.671357] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5ae5d992b3eb099b2d19935","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5ae5d992b3eb099b2d19935: start","job_status":"start","queue_duration_s":3.128343,"scheduling_latency_s":3.128363} gitlab # [ 252.684283] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5ae5d992b3eb099b2d19935","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5ae5d992b3eb099b2d19935: done: 0.016469 sec","job_status":"done","queue_duration_s":3.128343,"scheduling_latency_s":3.128363,"redis_calls":13,"redis_duration_s":0.001701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016469,"completed_at":"2026-08-27T02:05:28.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.724518] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e18f891731979c6868d1083d","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e18f891731979c6868d1083d: start","job_status":"start","queue_duration_s":3.146102,"scheduling_latency_s":3.146121} gitlab # [ 252.737233] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e18f891731979c6868d1083d","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e18f891731979c6868d1083d: done: 0.01633 sec","job_status":"done","queue_duration_s":3.146102,"scheduling_latency_s":3.146121,"redis_calls":13,"redis_duration_s":0.001697,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01633,"completed_at":"2026-08-27T02:05:28.143Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.777219] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e8540a6dd76f439a36a791f","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e8540a6dd76f439a36a791f: start","job_status":"start","queue_duration_s":3.163734,"scheduling_latency_s":3.163754} gitlab # [ 252.789950] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e8540a6dd76f439a36a791f","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e8540a6dd76f439a36a791f: done: 0.016625 sec","job_status":"done","queue_duration_s":3.163734,"scheduling_latency_s":3.163754,"redis_calls":13,"redis_duration_s":0.001679,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016625,"completed_at":"2026-08-27T02:05:28.161Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.830815] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72eca58349322e36ff25d8db","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72eca58349322e36ff25d8db: start","job_status":"start","queue_duration_s":3.181704,"scheduling_latency_s":3.181723} gitlab # [ 252.843684] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72eca58349322e36ff25d8db","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72eca58349322e36ff25d8db: done: 0.016309 sec","job_status":"done","queue_duration_s":3.181704,"scheduling_latency_s":3.181723,"redis_calls":13,"redis_duration_s":0.001785,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016309,"completed_at":"2026-08-27T02:05:28.178Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.884007] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5744fce92c854534f5d5789b","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5744fce92c854534f5d5789b: start","job_status":"start","queue_duration_s":3.199374,"scheduling_latency_s":3.199395} gitlab # [ 252.896609] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5744fce92c854534f5d5789b","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5744fce92c854534f5d5789b: done: 0.016853 sec","job_status":"done","queue_duration_s":3.199374,"scheduling_latency_s":3.199395,"redis_calls":13,"redis_duration_s":0.0017469999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016853,"completed_at":"2026-08-27T02:05:28.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.936956] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96b3dc0defcb1f9fa338ec65","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96b3dc0defcb1f9fa338ec65: start","job_status":"start","queue_duration_s":3.217568,"scheduling_latency_s":3.217589} gitlab # [ 252.949638] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96b3dc0defcb1f9fa338ec65","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96b3dc0defcb1f9fa338ec65: done: 0.017036 sec","job_status":"done","queue_duration_s":3.217568,"scheduling_latency_s":3.217589,"redis_calls":13,"redis_duration_s":0.0017569999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017036,"completed_at":"2026-08-27T02:05:28.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.989834] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c585fd3093b9ded46122b939","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c585fd3093b9ded46122b939: start","job_status":"start","queue_duration_s":3.236401,"scheduling_latency_s":3.236422} gitlab # [ 253.003491] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c585fd3093b9ded46122b939","created_at":"2026-08-27T02:05:24.298Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-6dac109c9447405d","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6910558ab1604476f21ed10cf2fcc73c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:24.980Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c585fd3093b9ded46122b939: done: 0.016617 sec","job_status":"done","queue_duration_s":3.236401,"scheduling_latency_s":3.236422,"redis_calls":13,"redis_duration_s":0.001739,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016617,"completed_at":"2026-08-27T02:05:28.233Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.044162] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1ebb1ae4aaa87efdada5489b","created_at":"2026-08-27T02:05:25.049Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"986bd1b44815c8b7c9bf030b7521806c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.074Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1ebb1ae4aaa87efdada5489b: start","job_status":"start","queue_duration_s":3.161287,"scheduling_latency_s":3.161308} gitlab # [ 253.059028] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.249Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 253.063021] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1ebb1ae4aaa87efdada5489b","created_at":"2026-08-27T02:05:25.049Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"986bd1b44815c8b7c9bf030b7521806c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.074Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1ebb1ae4aaa87efdada5489b: done: 0.022198 sec","job_status":"done","queue_duration_s":3.161287,"scheduling_latency_s":3.161308,"redis_calls":10,"redis_duration_s":0.001685,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029516,"duration_s":0.022198,"completed_at":"2026-08-27T02:05:28.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.106774] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"540a0831dc136d9e2105d495","created_at":"2026-08-27T02:05:25.103Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb51a6372ce257f18341f15c3f304327","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.190Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-540a0831dc136d9e2105d495: start","job_status":"start","queue_duration_s":3.067778,"scheduling_latency_s":3.067796} gitlab # [ 253.122021] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.271Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 253.125552] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.278Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"540a0831dc136d9e2105d495","created_at":"2026-08-27T02:05:25.103Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb51a6372ce257f18341f15c3f304327","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.190Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-540a0831dc136d9e2105d495: done: 0.019603 sec","job_status":"done","queue_duration_s":3.067778,"scheduling_latency_s":3.067796,"redis_calls":10,"redis_duration_s":0.001361,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024266,"duration_s":0.019603,"completed_at":"2026-08-27T02:05:28.278Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.169142] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"6799183fd08cd6cdee6239bb","created_at":"2026-08-27T02:05:25.253Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4adf37797fffa238f138c146fd10add7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.276Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-6799183fd08cd6cdee6239bb: start","job_status":"start","queue_duration_s":3.002625,"scheduling_latency_s":3.002642} gitlab # [ 253.184088] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.291Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 253.187283] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"6799183fd08cd6cdee6239bb","created_at":"2026-08-27T02:05:25.253Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4adf37797fffa238f138c146fd10add7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.276Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-6799183fd08cd6cdee6239bb: done: 0.018721 sec","job_status":"done","queue_duration_s":3.002625,"scheduling_latency_s":3.002642,"redis_calls":10,"redis_duration_s":0.001311,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023014,"duration_s":0.018721,"completed_at":"2026-08-27T02:05:28.298Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.232109] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.299Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"acf34edc6dd37d79b5237ac7","created_at":"2026-08-27T02:05:25.346Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"534b9fd63a3165b6d4c23e793801abdc","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.374Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-acf34edc6dd37d79b5237ac7: start","job_status":"start","queue_duration_s":2.924839,"scheduling_latency_s":2.924856} gitlab # [ 253.246530] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"acf34edc6dd37d79b5237ac7","created_at":"2026-08-27T02:05:25.346Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"534b9fd63a3165b6d4c23e793801abdc","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.374Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-acf34edc6dd37d79b5237ac7: done: 0.020112 sec","job_status":"done","queue_duration_s":2.924839,"scheduling_latency_s":2.924856,"redis_calls":11,"redis_duration_s":0.0018700000000000001,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000279,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002429,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.020112,"completed_at":"2026-08-27T02:05:28.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.291735] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.321Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b40771ef472151f1d8b04ab3","created_at":"2026-08-27T02:05:25.434Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55ee1138f7642f88d8ce6334e6acacbf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.450Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b40771ef472151f1d8b04ab3: start","job_status":"start","queue_duration_s":2.87031,"scheduling_latency_s":2.870329} gitlab # [ 253.306911] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b40771ef472151f1d8b04ab3","created_at":"2026-08-27T02:05:25.434Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55ee1138f7642f88d8ce6334e6acacbf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.450Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b40771ef472151f1d8b04ab3: done: 0.022153 sec","job_status":"done","queue_duration_s":2.87031,"scheduling_latency_s":2.870329,"redis_calls":12,"redis_duration_s":0.0016539999999999999,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002415,"duration_s":0.022153,"completed_at":"2026-08-27T02:05:28.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.351202] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"67c312fbd870fd110e6a8aed","created_at":"2026-08-27T02:05:25.469Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6c17710e429c891d79b80ef0c98df4e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.488Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-67c312fbd870fd110e6a8aed: start","job_status":"start","queue_duration_s":2.856941,"scheduling_latency_s":2.85696} gitlab # [ 253.366113] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"67c312fbd870fd110e6a8aed","created_at":"2026-08-27T02:05:25.469Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6c17710e429c891d79b80ef0c98df4e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.488Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-67c312fbd870fd110e6a8aed: done: 0.022918 sec","job_status":"done","queue_duration_s":2.856941,"scheduling_latency_s":2.85696,"redis_calls":10,"redis_duration_s":0.001359,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000312,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023002,"duration_s":0.022918,"completed_at":"2026-08-27T02:05:28.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000555,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.410043] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.369Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b612e53037459db672dddc54","created_at":"2026-08-27T02:05:25.834Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-aa3567cd61824bc0","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.865Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b612e53037459db672dddc54: start","job_status":"start","queue_duration_s":2.503444,"scheduling_latency_s":2.503463} gitlab # [ 253.422665] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.387Z","message":"No existing merge request to be cleaned up."} gitlab # [ 253.424381] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.404Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b612e53037459db672dddc54","created_at":"2026-08-27T02:05:25.834Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-aa3567cd61824bc0","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.865Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b612e53037459db672dddc54: done: 0.035445 sec","job_status":"done","queue_duration_s":2.503444,"scheduling_latency_s":2.503463,"redis_calls":13,"redis_duration_s":0.0018,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000864,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023696,"duration_s":0.035445,"completed_at":"2026-08-27T02:05:28.404Z","load_balancing_strategy":"primary","db_duration_s":0.005819,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.465500] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.405Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9ee4e9518a646ce356b6a762","created_at":"2026-08-27T02:05:25.834Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-aa3567cd61824bc0","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.865Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9ee4e9518a646ce356b6a762: start","job_status":"start","queue_duration_s":2.540064,"scheduling_latency_s":2.540082} gitlab # [ 253.478094] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.419Z","message":"No existing merge request to be cleaned up."} gitlab # [ 253.479786] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.428Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9ee4e9518a646ce356b6a762","created_at":"2026-08-27T02:05:25.834Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-aa3567cd61824bc0","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.865Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9ee4e9518a646ce356b6a762: done: 0.022962 sec","job_status":"done","queue_duration_s":2.540064,"scheduling_latency_s":2.540082,"redis_calls":10,"redis_duration_s":0.001467,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000848,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022962,"completed_at":"2026-08-27T02:05:28.428Z","load_balancing_strategy":"primary","db_duration_s":0.002912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.519585] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.430Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d4474d5749f2011b48aa679e","created_at":"2026-08-27T02:05:25.834Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-aa3567cd61824bc0","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.865Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d4474d5749f2011b48aa679e: start","job_status":"start","queue_duration_s":2.564612,"scheduling_latency_s":2.564631} gitlab # [ 253.532465] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.443Z","message":"No existing merge request to be cleaned up."} gitlab # [ 253.534172] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.452Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d4474d5749f2011b48aa679e","created_at":"2026-08-27T02:05:25.834Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-aa3567cd61824bc0","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.865Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d4474d5749f2011b48aa679e: done: 0.021745 sec","job_status":"done","queue_duration_s":2.564612,"scheduling_latency_s":2.564631,"redis_calls":10,"redis_duration_s":0.0014759999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000856,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021745,"completed_at":"2026-08-27T02:05:28.452Z","load_balancing_strategy":"primary","db_duration_s":0.00282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.573941] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.454Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b067ba4eedf2486f3e08af47","created_at":"2026-08-27T02:05:25.834Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-aa3567cd61824bc0","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.865Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b067ba4eedf2486f3e08af47: start","job_status":"start","queue_duration_s":2.588224,"scheduling_latency_s":2.588245} gitlab # [ 253.586585] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.466Z","message":"No existing merge request to be cleaned up."} gitlab # [ 253.588279] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:05:28.476Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b067ba4eedf2486f3e08af47","created_at":"2026-08-27T02:05:25.834Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-aa3567cd61824bc0","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4185b08cfa35f752d647b5979023c6f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:05:25.865Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b067ba4eedf2486f3e08af47: done: 0.021997 sec","job_status":"done","queue_duration_s":2.588224,"scheduling_latency_s":2.588245,"redis_calls":10,"redis_duration_s":0.002131,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001497,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021997,"completed_at":"2026-08-27T02:05:28.476Z","load_balancing_strategy":"primary","db_duration_s":0.002772,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.745251] setup-gitlab-start[3061]: W, [2026-08-27T02:06:01.615571 #3061] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 285.322948] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:02.192Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"72f70748e7724442704665be","created_at":"2026-08-27T02:06:02.169Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cbd6c83a03a950f3f05174e5b9340043","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.188Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-72f70748e7724442704665be: start","job_status":"start","queue_duration_s":0.003737,"scheduling_latency_s":0.003795} gitlab # [ 285.448860] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:02.319Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"72f70748e7724442704665be","created_at":"2026-08-27T02:06:02.169Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cbd6c83a03a950f3f05174e5b9340043","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.188Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-72f70748e7724442704665be: done: 0.126532 sec","job_status":"done","queue_duration_s":0.003737,"scheduling_latency_s":0.003795,"redis_calls":9,"redis_duration_s":0.019781999999999998,"redis_read_bytes":137,"redis_write_bytes":779,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002409,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":68,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.015766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001607,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.126532,"completed_at":"2026-08-27T02:06:02.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.533957] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:02.324Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"88c40db16a6ddb4b69941bac","created_at":"2026-08-27T02:06:02.216Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.240Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-88c40db16a6ddb4b69941bac: start","job_status":"start","queue_duration_s":0.084099,"scheduling_latency_s":0.084158} gitlab # [ 286.176256] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"88c40db16a6ddb4b69941bac","created_at":"2026-08-27T02:06:02.216Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.240Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-88c40db16a6ddb4b69941bac: done: 0.721083 sec","job_status":"done","queue_duration_s":0.084099,"scheduling_latency_s":0.084158,"redis_calls":60,"redis_duration_s":0.19825800000000002,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.004583,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.191251,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002424,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.305627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.721083,"completed_at":"2026-08-27T02:06:03.045Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.206469] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"8883334a31f6807d66d713ee","created_at":"2026-08-27T02:06:02.270Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54c397cd7d62842b21443b61ae6e63a3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.377Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-8883334a31f6807d66d713ee: start","job_status":"start","queue_duration_s":0.698537,"scheduling_latency_s":0.698603} gitlab # [ 286.483713] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.352Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"8883334a31f6807d66d713ee","created_at":"2026-08-27T02:06:02.270Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54c397cd7d62842b21443b61ae6e63a3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.377Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-8883334a31f6807d66d713ee: done: 0.277255 sec","job_status":"done","queue_duration_s":0.698537,"scheduling_latency_s":0.698603,"redis_calls":13,"redis_duration_s":0.033347,"redis_read_bytes":139,"redis_write_bytes":1611,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.007148,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":386,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.012055,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.055,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.055,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00337318,"exclusive_lock_hold_duration_s":0.08045994,"duration_s":0.277255,"completed_at":"2026-08-27T02:06:03.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.066432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.569889] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.361Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"10b0b1c0610f59e9f3ff0f2a","created_at":"2026-08-27T02:06:02.397Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46adad8512c63f702c7291851042a9a4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.411Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-10b0b1c0610f59e9f3ff0f2a: start","job_status":"start","queue_duration_s":0.950373,"scheduling_latency_s":0.950442} gitlab # [ 286.613920] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.484Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"10b0b1c0610f59e9f3ff0f2a","created_at":"2026-08-27T02:06:02.397Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46adad8512c63f702c7291851042a9a4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.411Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-10b0b1c0610f59e9f3ff0f2a: done: 0.123229 sec","job_status":"done","queue_duration_s":0.950373,"scheduling_latency_s":0.950442,"redis_calls":9,"redis_duration_s":0.007006,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004377,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00170448,"exclusive_lock_hold_duration_s":0.03752746,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.123229,"completed_at":"2026-08-27T02:06:03.484Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.688804] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.540Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"2e2a56adf6bd3c448902347d","created_at":"2026-08-27T02:06:02.442Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12612af9643de60d6703ef104387183d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-27T02:06:02.453Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-2e2a56adf6bd3c448902347d: start","job_status":"start","queue_duration_s":1.08627,"scheduling_latency_s":1.086339} gitlab # [ 286.711200] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.582Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 286.758296] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"2e2a56adf6bd3c448902347d","created_at":"2026-08-27T02:06:02.442Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12612af9643de60d6703ef104387183d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-27T02:06:02.453Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-2e2a56adf6bd3c448902347d: done: 0.089037 sec","job_status":"done","queue_duration_s":1.08627,"scheduling_latency_s":1.086339,"redis_calls":7,"redis_duration_s":0.010095,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.089037,"completed_at":"2026-08-27T02:06:03.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.835051] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"86a42b60c0a2bab86d002239","created_at":"2026-08-27T02:06:02.478Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62bddcce8edef9966c145288e9cb5d64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.489Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-86a42b60c0a2bab86d002239: start","job_status":"start","queue_duration_s":1.142909,"scheduling_latency_s":1.142959} gitlab # [ 286.893336] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"86a42b60c0a2bab86d002239","created_at":"2026-08-27T02:06:02.478Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62bddcce8edef9966c145288e9cb5d64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.489Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-86a42b60c0a2bab86d002239: done: 0.131354 sec","job_status":"done","queue_duration_s":1.142909,"scheduling_latency_s":1.142959,"redis_calls":9,"redis_duration_s":0.004805,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002121,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.039,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00145566,"exclusive_lock_hold_duration_s":0.0963267,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.131354,"completed_at":"2026-08-27T02:06:03.763Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.07268,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.951810] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.770Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dd94d330aa806866379d5c72","created_at":"2026-08-27T02:06:02.516Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fffa098bea844e23df6eaaf89ce73d5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.530Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dd94d330aa806866379d5c72: start","job_status":"start","queue_duration_s":1.240305,"scheduling_latency_s":1.240361} gitlab # [ 286.966797] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.808Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dd94d330aa806866379d5c72","created_at":"2026-08-27T02:06:02.516Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fffa098bea844e23df6eaaf89ce73d5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.530Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dd94d330aa806866379d5c72: done: 0.038204 sec","job_status":"done","queue_duration_s":1.240305,"scheduling_latency_s":1.240361,"redis_calls":7,"redis_duration_s":0.002998,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002199,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000799,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038204,"completed_at":"2026-08-27T02:06:03.808Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003143,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.009768] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c1e6b807fdc179bfcc3b3a02","created_at":"2026-08-27T02:06:02.558Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67b20448961ecc558257aadbcd3adefe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-27T02:06:02.570Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c1e6b807fdc179bfcc3b3a02: start","job_status":"start","queue_duration_s":1.241248,"scheduling_latency_s":1.241292} gitlab # [ 287.023672] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c1e6b807fdc179bfcc3b3a02","created_at":"2026-08-27T02:06:02.558Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67b20448961ecc558257aadbcd3adefe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-27T02:06:02.570Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c1e6b807fdc179bfcc3b3a02: done: 0.044742 sec","job_status":"done","queue_duration_s":1.241248,"scheduling_latency_s":1.241292,"redis_calls":8,"redis_duration_s":0.0037430000000000002,"redis_read_bytes":133,"redis_write_bytes":811,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000339,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000753,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044742,"completed_at":"2026-08-27T02:06:03.856Z","load_balancing_strategy":"primary","db_duration_s":0.003948,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.066232] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"195692deecaeb38e298d01eb","created_at":"2026-08-27T02:06:02.598Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e9ac4bea43305ebf98c3aaeeb7dfe71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.610Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-195692deecaeb38e298d01eb: start","job_status":"start","queue_duration_s":1.24914,"scheduling_latency_s":1.249179} gitlab # [ 287.080557] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"195692deecaeb38e298d01eb","created_at":"2026-08-27T02:06:02.598Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e9ac4bea43305ebf98c3aaeeb7dfe71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.610Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-195692deecaeb38e298d01eb: done: 0.029302 sec","job_status":"done","queue_duration_s":1.24914,"scheduling_latency_s":1.249179,"redis_calls":9,"redis_duration_s":0.003157,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001634,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001523,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00092312,"exclusive_lock_hold_duration_s":0.00551744,"duration_s":0.029302,"completed_at":"2026-08-27T02:06:03.888Z","load_balancing_strategy":"primary","db_duration_s":0.002083,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.123657] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.890Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2b1abf559647fc116329d4e7","created_at":"2026-08-27T02:06:02.643Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.656Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2b1abf559647fc116329d4e7: start","job_status":"start","queue_duration_s":1.234188,"scheduling_latency_s":1.234214} gitlab # [ 287.137712] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.931Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2b1abf559647fc116329d4e7","created_at":"2026-08-27T02:06:02.643Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.656Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2b1abf559647fc116329d4e7: done: 0.041081 sec","job_status":"done","queue_duration_s":1.234188,"scheduling_latency_s":1.234214,"redis_calls":14,"redis_duration_s":0.004756,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000462,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003105,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041081,"completed_at":"2026-08-27T02:06:03.931Z","load_balancing_strategy":"primary","db_duration_s":0.002169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.179369] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.935Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"653b566cb9bab05ed74b7067","created_at":"2026-08-27T02:06:02.698Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aca97049c8c54c381050444883affba7","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.736Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-653b566cb9bab05ed74b7067: start","job_status":"start","queue_duration_s":1.19939,"scheduling_latency_s":1.199441} gitlab # [ 287.192991] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.977Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"653b566cb9bab05ed74b7067","created_at":"2026-08-27T02:06:02.698Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aca97049c8c54c381050444883affba7","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.736Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-653b566cb9bab05ed74b7067: done: 0.042065 sec","job_status":"done","queue_duration_s":1.19939,"scheduling_latency_s":1.199441,"redis_calls":12,"redis_duration_s":0.003884,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000848,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00064,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000606,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045286,"duration_s":0.042065,"completed_at":"2026-08-27T02:06:03.977Z","load_balancing_strategy":"primary","db_duration_s":0.001005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.237016] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:03.979Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"d2d0b2a993f31ebaeb1cf6fd","created_at":"2026-08-27T02:06:02.795Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"360be862e61cfef2ac8d3ff5790af368","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.817Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-d2d0b2a993f31ebaeb1cf6fd: start","job_status":"start","queue_duration_s":1.161785,"scheduling_latency_s":1.161817} gitlab # [ 287.252161] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"d2d0b2a993f31ebaeb1cf6fd","created_at":"2026-08-27T02:06:02.795Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"360be862e61cfef2ac8d3ff5790af368","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.817Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-d2d0b2a993f31ebaeb1cf6fd: done: 0.031382 sec","job_status":"done","queue_duration_s":1.161785,"scheduling_latency_s":1.161817,"redis_calls":10,"redis_duration_s":0.002538,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001468,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00772482,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.031382,"completed_at":"2026-08-27T02:06:04.010Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004316,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 287.296827] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.012Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"014b126c60981b56fcd08649","created_at":"2026-08-27T02:06:02.855Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3677548cb5935052d846ec256f6e8009","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.868Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-014b126c60981b56fcd08649: start","job_status":"start","queue_duration_s":1.144101,"scheduling_latency_s":1.144132} gitlab # [ 287.627878] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.497Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"014b126c60981b56fcd08649","created_at":"2026-08-27T02:06:02.855Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3677548cb5935052d846ec256f6e8009","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.868Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-014b126c60981b56fcd08649: done: 0.485055 sec","job_status":"done","queue_duration_s":1.144101,"scheduling_latency_s":1.144132,"redis_calls":562,"redis_duration_s":0.12582000000000002,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.124587,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.415183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.485055,"completed_at":"2026-08-27T02:06:04.497Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.717304] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.501Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6fbd5da59f60f7380a18f2a7","created_at":"2026-08-27T02:06:02.939Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8091a91133704d9ca37032a2ecb92eae","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.952Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6fbd5da59f60f7380a18f2a7: start","job_status":"start","queue_duration_s":1.548731,"scheduling_latency_s":1.548798} gitlab # [ 287.740687] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6fbd5da59f60f7380a18f2a7","created_at":"2026-08-27T02:06:02.939Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8091a91133704d9ca37032a2ecb92eae","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.952Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6fbd5da59f60f7380a18f2a7: done: 0.079448 sec","job_status":"done","queue_duration_s":1.548731,"scheduling_latency_s":1.548798,"redis_calls":9,"redis_duration_s":0.0038669999999999998,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001875,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00125382,"exclusive_lock_hold_duration_s":0.02985602,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.079448,"completed_at":"2026-08-27T02:06:04.580Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.799274] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1a0522553b7ad3bb84648ec","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1a0522553b7ad3bb84648ec: start","job_status":"start","queue_duration_s":1.605034,"scheduling_latency_s":1.605104} gitlab # [ 287.812638] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1a0522553b7ad3bb84648ec","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1a0522553b7ad3bb84648ec: done: 0.045003 sec","job_status":"done","queue_duration_s":1.605034,"scheduling_latency_s":1.605104,"redis_calls":13,"redis_duration_s":0.004622,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.045003,"completed_at":"2026-08-27T02:06:04.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.852995] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dee3d9bad31abba268963a23","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dee3d9bad31abba268963a23: start","job_status":"start","queue_duration_s":1.652577,"scheduling_latency_s":1.652623} gitlab # [ 287.865676] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dee3d9bad31abba268963a23","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dee3d9bad31abba268963a23: done: 0.038493 sec","job_status":"done","queue_duration_s":1.652577,"scheduling_latency_s":1.652623,"redis_calls":13,"redis_duration_s":0.007312000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005825,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038493,"completed_at":"2026-08-27T02:06:04.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.906777] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ccf37e58dae388d0a70911c","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ccf37e58dae388d0a70911c: start","job_status":"start","queue_duration_s":1.693809,"scheduling_latency_s":1.693859} gitlab # [ 287.920099] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ccf37e58dae388d0a70911c","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ccf37e58dae388d0a70911c: done: 0.0303 sec","job_status":"done","queue_duration_s":1.693809,"scheduling_latency_s":1.693859,"redis_calls":13,"redis_duration_s":0.003155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001138,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0303,"completed_at":"2026-08-27T02:06:04.702Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.961730] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0fffd210f38e5386dcdecaa","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0fffd210f38e5386dcdecaa: start","job_status":"start","queue_duration_s":1.726293,"scheduling_latency_s":1.726332} gitlab # [ 287.974910] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0fffd210f38e5386dcdecaa","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0fffd210f38e5386dcdecaa: done: 0.025539 sec","job_status":"done","queue_duration_s":1.726293,"scheduling_latency_s":1.726332,"redis_calls":13,"redis_duration_s":0.002674,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001043,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001631,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025539,"completed_at":"2026-08-27T02:06:04.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.015274] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c7543286f2fe8c379f2ee74","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c7543286f2fe8c379f2ee74: start","job_status":"start","queue_duration_s":1.753868,"scheduling_latency_s":1.753908} gitlab # [ 288.028266] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c7543286f2fe8c379f2ee74","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c7543286f2fe8c379f2ee74: done: 0.024063 sec","job_status":"done","queue_duration_s":1.753868,"scheduling_latency_s":1.753908,"redis_calls":13,"redis_duration_s":0.002533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001505,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024063,"completed_at":"2026-08-27T02:06:04.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.069092] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90d071708e2b8ddb49321d57","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90d071708e2b8ddb49321d57: start","job_status":"start","queue_duration_s":1.779773,"scheduling_latency_s":1.77981} gitlab # [ 288.081927] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90d071708e2b8ddb49321d57","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90d071708e2b8ddb49321d57: done: 0.025749 sec","job_status":"done","queue_duration_s":1.779773,"scheduling_latency_s":1.77981,"redis_calls":13,"redis_duration_s":0.002742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001688,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025749,"completed_at":"2026-08-27T02:06:04.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.122153] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb2a7bc4a6c52d2acf2db471","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb2a7bc4a6c52d2acf2db471: start","job_status":"start","queue_duration_s":1.807855,"scheduling_latency_s":1.807915} gitlab # [ 288.136094] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb2a7bc4a6c52d2acf2db471","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb2a7bc4a6c52d2acf2db471: done: 0.025397 sec","job_status":"done","queue_duration_s":1.807855,"scheduling_latency_s":1.807915,"redis_calls":13,"redis_duration_s":0.002704,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025397,"completed_at":"2026-08-27T02:06:04.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.176396] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a502a38a5549b4ebfce5cb","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a502a38a5549b4ebfce5cb: start","job_status":"start","queue_duration_s":1.835156,"scheduling_latency_s":1.835192} gitlab # [ 288.189140] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a502a38a5549b4ebfce5cb","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a502a38a5549b4ebfce5cb: done: 0.024543 sec","job_status":"done","queue_duration_s":1.835156,"scheduling_latency_s":1.835192,"redis_calls":13,"redis_duration_s":0.002633,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001628,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024543,"completed_at":"2026-08-27T02:06:04.838Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.229381] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53083616a9a84dcaa4dc3943","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53083616a9a84dcaa4dc3943: start","job_status":"start","queue_duration_s":1.861683,"scheduling_latency_s":1.861726} gitlab # [ 288.242128] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53083616a9a84dcaa4dc3943","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53083616a9a84dcaa4dc3943: done: 0.024722 sec","job_status":"done","queue_duration_s":1.861683,"scheduling_latency_s":1.861726,"redis_calls":13,"redis_duration_s":0.0024739999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001478,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024722,"completed_at":"2026-08-27T02:06:04.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.282605] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"073cb6b073ee1187d7fe2b2a","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-073cb6b073ee1187d7fe2b2a: start","job_status":"start","queue_duration_s":1.888324,"scheduling_latency_s":1.888356} gitlab # [ 288.295697] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"073cb6b073ee1187d7fe2b2a","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-073cb6b073ee1187d7fe2b2a: done: 0.024152 sec","job_status":"done","queue_duration_s":1.888324,"scheduling_latency_s":1.888356,"redis_calls":13,"redis_duration_s":0.002507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001523,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024152,"completed_at":"2026-08-27T02:06:04.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.337342] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5383c060099528500759a51","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5383c060099528500759a51: start","job_status":"start","queue_duration_s":1.914556,"scheduling_latency_s":1.914594} gitlab # [ 288.350131] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5383c060099528500759a51","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5383c060099528500759a51: done: 0.024267 sec","job_status":"done","queue_duration_s":1.914556,"scheduling_latency_s":1.914594,"redis_calls":13,"redis_duration_s":0.002526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024267,"completed_at":"2026-08-27T02:06:04.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.390503] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92f0053402c6ecf29c31d261","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92f0053402c6ecf29c31d261: start","job_status":"start","queue_duration_s":1.940739,"scheduling_latency_s":1.940776} gitlab # [ 288.403385] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92f0053402c6ecf29c31d261","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92f0053402c6ecf29c31d261: done: 0.024249 sec","job_status":"done","queue_duration_s":1.940739,"scheduling_latency_s":1.940776,"redis_calls":13,"redis_duration_s":0.002516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001509,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024249,"completed_at":"2026-08-27T02:06:04.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.444832] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:04.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0552ae046bacf74ac2905dfd","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0552ae046bacf74ac2905dfd: start","job_status":"start","queue_duration_s":1.966943,"scheduling_latency_s":1.966979} gitlab # [ 288.457714] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0552ae046bacf74ac2905dfd","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0552ae046bacf74ac2905dfd: done: 0.307574 sec","job_status":"done","queue_duration_s":1.966943,"scheduling_latency_s":1.966979,"redis_calls":13,"redis_duration_s":0.002803,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.307574,"completed_at":"2026-08-27T02:06:05.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.497955] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e98bc47be79880dd6c5ecb13","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e98bc47be79880dd6c5ecb13: start","job_status":"start","queue_duration_s":2.282062,"scheduling_latency_s":2.282097} gitlab # [ 288.559804] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e98bc47be79880dd6c5ecb13","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e98bc47be79880dd6c5ecb13: done: 0.170068 sec","job_status":"done","queue_duration_s":2.282062,"scheduling_latency_s":2.282097,"redis_calls":13,"redis_duration_s":0.073382,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.061306,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.170068,"completed_at":"2026-08-27T02:06:05.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.602989] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e102e077cb3f898a2a1978b3","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e102e077cb3f898a2a1978b3: start","job_status":"start","queue_duration_s":2.482925,"scheduling_latency_s":2.482986} gitlab # [ 288.661204] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e102e077cb3f898a2a1978b3","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e102e077cb3f898a2a1978b3: done: 0.069391 sec","job_status":"done","queue_duration_s":2.482925,"scheduling_latency_s":2.482986,"redis_calls":13,"redis_duration_s":0.016713,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005147,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.011566,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.069391,"completed_at":"2026-08-27T02:06:05.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.703710] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc404f0c58d34da89493a6e5","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc404f0c58d34da89493a6e5: start","job_status":"start","queue_duration_s":2.562857,"scheduling_latency_s":2.562913} gitlab # [ 288.779419] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc404f0c58d34da89493a6e5","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc404f0c58d34da89493a6e5: done: 0.10809 sec","job_status":"done","queue_duration_s":2.562857,"scheduling_latency_s":2.562913,"redis_calls":13,"redis_duration_s":0.013835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.10809,"completed_at":"2026-08-27T02:06:05.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.862988] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58f110d959bb5ccb116c4fa7","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58f110d959bb5ccb116c4fa7: start","job_status":"start","queue_duration_s":2.679207,"scheduling_latency_s":2.679263} gitlab # [ 288.879796] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58f110d959bb5ccb116c4fa7","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58f110d959bb5ccb116c4fa7: done: 0.092027 sec","job_status":"done","queue_duration_s":2.679207,"scheduling_latency_s":2.679263,"redis_calls":13,"redis_duration_s":0.023787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.015248,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.092027,"completed_at":"2026-08-27T02:06:05.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 288.926978] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aad6994d5c8632cea3bee29","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aad6994d5c8632cea3bee29: start","job_status":"start","queue_duration_s":2.783283,"scheduling_latency_s":2.783345} gitlab # [ 289.052142] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:05.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aad6994d5c8632cea3bee29","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aad6994d5c8632cea3bee29: done: 0.160292 sec","job_status":"done","queue_duration_s":2.783283,"scheduling_latency_s":2.783345,"redis_calls":13,"redis_duration_s":0.078148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.01072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.067428,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.160292,"completed_at":"2026-08-27T02:06:05.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.140991] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbf23654d09716106ffa1f00","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbf23654d09716106ffa1f00: start","job_status":"start","queue_duration_s":3.031232,"scheduling_latency_s":3.031305} gitlab # [ 289.295751] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbf23654d09716106ffa1f00","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbf23654d09716106ffa1f00: done: 0.156107 sec","job_status":"done","queue_duration_s":3.031232,"scheduling_latency_s":3.031305,"redis_calls":13,"redis_duration_s":0.026253,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007375,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.018878,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.156107,"completed_at":"2026-08-27T02:06:06.166Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.410958] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d60bfee9d75d257420d0c153","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d60bfee9d75d257420d0c153: start","job_status":"start","queue_duration_s":3.302908,"scheduling_latency_s":3.302975} gitlab # [ 289.554116] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d60bfee9d75d257420d0c153","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d60bfee9d75d257420d0c153: done: 0.142919 sec","job_status":"done","queue_duration_s":3.302908,"scheduling_latency_s":3.302975,"redis_calls":13,"redis_duration_s":0.023048,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006753,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.016295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.142919,"completed_at":"2026-08-27T02:06:06.424Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.635284] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32ae44aa5ab541cc9990fb9e","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32ae44aa5ab541cc9990fb9e: start","job_status":"start","queue_duration_s":3.47195,"scheduling_latency_s":3.472012} gitlab # [ 289.674111] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32ae44aa5ab541cc9990fb9e","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32ae44aa5ab541cc9990fb9e: done: 0.093178 sec","job_status":"done","queue_duration_s":3.47195,"scheduling_latency_s":3.472012,"redis_calls":13,"redis_duration_s":0.033519999999999994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.023998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.093178,"completed_at":"2026-08-27T02:06:06.543Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.737087] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33dbd7caf2771f00947728fa","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33dbd7caf2771f00947728fa: start","job_status":"start","queue_duration_s":3.5794,"scheduling_latency_s":3.579463} gitlab # [ 289.794421] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33dbd7caf2771f00947728fa","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33dbd7caf2771f00947728fa: done: 0.106961 sec","job_status":"done","queue_duration_s":3.5794,"scheduling_latency_s":3.579463,"redis_calls":13,"redis_duration_s":0.035570000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.025644,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.106961,"completed_at":"2026-08-27T02:06:06.665Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.859264] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be05b57fec1592e6b93a87fa","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be05b57fec1592e6b93a87fa: start","job_status":"start","queue_duration_s":3.701463,"scheduling_latency_s":3.701524} gitlab # [ 289.907137] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be05b57fec1592e6b93a87fa","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be05b57fec1592e6b93a87fa: done: 0.097728 sec","job_status":"done","queue_duration_s":3.701463,"scheduling_latency_s":3.701524,"redis_calls":13,"redis_duration_s":0.03157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.01239,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.097728,"completed_at":"2026-08-27T02:06:06.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 289.967265] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c813d19ae1a9c8b5aaead670","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c813d19ae1a9c8b5aaead670: start","job_status":"start","queue_duration_s":3.812666,"scheduling_latency_s":3.812729} gitlab # [ 290.032231] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c813d19ae1a9c8b5aaead670","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c813d19ae1a9c8b5aaead670: done: 0.109427 sec","job_status":"done","queue_duration_s":3.812666,"scheduling_latency_s":3.812729,"redis_calls":13,"redis_duration_s":0.031398999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.023464,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.109427,"completed_at":"2026-08-27T02:06:06.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 290.118287] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:06.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14a82a4fc7d1c7a6dc2d81f6","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14a82a4fc7d1c7a6dc2d81f6: start","job_status":"start","queue_duration_s":4.010789,"scheduling_latency_s":4.010819} gitlab # [ 290.197567] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14a82a4fc7d1c7a6dc2d81f6","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14a82a4fc7d1c7a6dc2d81f6: done: 0.078969 sec","job_status":"done","queue_duration_s":4.010789,"scheduling_latency_s":4.010819,"redis_calls":13,"redis_duration_s":0.027993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.022033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.078969,"completed_at":"2026-08-27T02:06:07.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 290.272907] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a6d6fd6126ffea48f354533","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a6d6fd6126ffea48f354533: start","job_status":"start","queue_duration_s":4.094118,"scheduling_latency_s":4.094149} gitlab # [ 290.287802] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a6d6fd6126ffea48f354533","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a6d6fd6126ffea48f354533: done: 0.072514 sec","job_status":"done","queue_duration_s":4.094118,"scheduling_latency_s":4.094149,"redis_calls":13,"redis_duration_s":0.003136,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.072514,"completed_at":"2026-08-27T02:06:07.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 290.331309] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df392aac43446583a9a42a19","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df392aac43446583a9a42a19: start","job_status":"start","queue_duration_s":4.173979,"scheduling_latency_s":4.174027} gitlab # [ 290.348618] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df392aac43446583a9a42a19","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df392aac43446583a9a42a19: done: 0.066091 sec","job_status":"done","queue_duration_s":4.173979,"scheduling_latency_s":4.174027,"redis_calls":13,"redis_duration_s":0.004064,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001377,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002687,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.066091,"completed_at":"2026-08-27T02:06:07.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 290.393453] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a14534d6a325cdd3aa965b2","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a14534d6a325cdd3aa965b2: start","job_status":"start","queue_duration_s":4.245989,"scheduling_latency_s":4.246034} gitlab # [ 290.415868] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a14534d6a325cdd3aa965b2","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a14534d6a325cdd3aa965b2: done: 0.061791 sec","job_status":"done","queue_duration_s":4.245989,"scheduling_latency_s":4.246034,"redis_calls":13,"redis_duration_s":0.006529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.061791,"completed_at":"2026-08-27T02:06:07.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 290.460018] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"268bb69842c7728294b70599","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-268bb69842c7728294b70599: start","job_status":"start","queue_duration_s":4.310833,"scheduling_latency_s":4.310875} gitlab # [ 290.479430] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"268bb69842c7728294b70599","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-268bb69842c7728294b70599: done: 0.061022 sec","job_status":"done","queue_duration_s":4.310833,"scheduling_latency_s":4.310875,"redis_calls":13,"redis_duration_s":0.004534,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001362,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.061022,"completed_at":"2026-08-27T02:06:07.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 290.562515] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b35040bf0a5caaf2eae4e6c","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b35040bf0a5caaf2eae4e6c: start","job_status":"start","queue_duration_s":4.454113,"scheduling_latency_s":4.454162} gitlab # [ 290.745049] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b35040bf0a5caaf2eae4e6c","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b35040bf0a5caaf2eae4e6c: done: 0.180447 sec","job_status":"done","queue_duration_s":4.454113,"scheduling_latency_s":4.454162,"redis_calls":13,"redis_duration_s":0.033538,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012461,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.021077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.180447,"completed_at":"2026-08-27T02:06:07.613Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 290.859900] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe813be3d01ce14c7ce5d7c4","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe813be3d01ce14c7ce5d7c4: start","job_status":"start","queue_duration_s":4.75173,"scheduling_latency_s":4.751794} gitlab # [ 291.014752] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe813be3d01ce14c7ce5d7c4","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe813be3d01ce14c7ce5d7c4: done: 0.154576 sec","job_status":"done","queue_duration_s":4.75173,"scheduling_latency_s":4.751794,"redis_calls":13,"redis_duration_s":0.023289999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.016717,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.154576,"completed_at":"2026-08-27T02:06:07.885Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.117054] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:07.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6965ce9462e9e14732a83c8","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6965ce9462e9e14732a83c8: start","job_status":"start","queue_duration_s":4.970325,"scheduling_latency_s":4.970369} gitlab # [ 291.247667] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6965ce9462e9e14732a83c8","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6965ce9462e9e14732a83c8: done: 0.168873 sec","job_status":"done","queue_duration_s":4.970325,"scheduling_latency_s":4.970369,"redis_calls":13,"redis_duration_s":0.025315999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.018466,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.168873,"completed_at":"2026-08-27T02:06:08.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.334939] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"815c6dcd83677786838c078d","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-815c6dcd83677786838c078d: start","job_status":"start","queue_duration_s":5.145921,"scheduling_latency_s":5.145982} gitlab # [ 291.351205] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"815c6dcd83677786838c078d","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-815c6dcd83677786838c078d: done: 0.057533 sec","job_status":"done","queue_duration_s":5.145921,"scheduling_latency_s":5.145982,"redis_calls":13,"redis_duration_s":0.004599,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.057533,"completed_at":"2026-08-27T02:06:08.182Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.393520] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fbb534a56a1d412335e5609","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fbb534a56a1d412335e5609: start","job_status":"start","queue_duration_s":5.206657,"scheduling_latency_s":5.206713} gitlab # [ 291.406505] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fbb534a56a1d412335e5609","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fbb534a56a1d412335e5609: done: 0.045946 sec","job_status":"done","queue_duration_s":5.206657,"scheduling_latency_s":5.206713,"redis_calls":13,"redis_duration_s":0.003869,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001398,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002471,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.045946,"completed_at":"2026-08-27T02:06:08.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.448374] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc24691a3e677e364a2eef5c","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc24691a3e677e364a2eef5c: start","job_status":"start","queue_duration_s":5.255268,"scheduling_latency_s":5.255314} gitlab # [ 291.461258] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc24691a3e677e364a2eef5c","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc24691a3e677e364a2eef5c: done: 0.030213 sec","job_status":"done","queue_duration_s":5.255268,"scheduling_latency_s":5.255314,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030213,"completed_at":"2026-08-27T02:06:08.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.502140] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"147ef673a27cb8a558a4170f","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-147ef673a27cb8a558a4170f: start","job_status":"start","queue_duration_s":5.287546,"scheduling_latency_s":5.287585} gitlab # [ 291.515311] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"147ef673a27cb8a558a4170f","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-147ef673a27cb8a558a4170f: done: 0.025135 sec","job_status":"done","queue_duration_s":5.287546,"scheduling_latency_s":5.287585,"redis_calls":13,"redis_duration_s":0.002509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025135,"completed_at":"2026-08-27T02:06:08.291Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.555755] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57c40a5dc87a4990ac46d950","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57c40a5dc87a4990ac46d950: start","job_status":"start","queue_duration_s":5.314595,"scheduling_latency_s":5.314632} gitlab # [ 291.568665] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57c40a5dc87a4990ac46d950","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57c40a5dc87a4990ac46d950: done: 0.02343 sec","job_status":"done","queue_duration_s":5.314595,"scheduling_latency_s":5.314632,"redis_calls":13,"redis_duration_s":0.002322,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001454,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02343,"completed_at":"2026-08-27T02:06:08.316Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.609945] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58a2d282b1ec81156034052d","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58a2d282b1ec81156034052d: start","job_status":"start","queue_duration_s":5.339784,"scheduling_latency_s":5.33982} gitlab # [ 291.622779] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58a2d282b1ec81156034052d","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58a2d282b1ec81156034052d: done: 0.022731 sec","job_status":"done","queue_duration_s":5.339784,"scheduling_latency_s":5.33982,"redis_calls":13,"redis_duration_s":0.002379,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001461,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022731,"completed_at":"2026-08-27T02:06:08.341Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.663381] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb66043e6df44c4706c2287b","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb66043e6df44c4706c2287b: start","job_status":"start","queue_duration_s":5.364298,"scheduling_latency_s":5.364334} gitlab # [ 291.676391] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb66043e6df44c4706c2287b","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb66043e6df44c4706c2287b: done: 0.023521 sec","job_status":"done","queue_duration_s":5.364298,"scheduling_latency_s":5.364334,"redis_calls":13,"redis_duration_s":0.00236,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001463,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023521,"completed_at":"2026-08-27T02:06:08.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.716971] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2818be105a5f4882acf361cb","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2818be105a5f4882acf361cb: start","job_status":"start","queue_duration_s":5.389718,"scheduling_latency_s":5.389757} gitlab # [ 291.729700] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2818be105a5f4882acf361cb","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2818be105a5f4882acf361cb: done: 0.022889 sec","job_status":"done","queue_duration_s":5.389718,"scheduling_latency_s":5.389757,"redis_calls":13,"redis_duration_s":0.002339,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001447,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022889,"completed_at":"2026-08-27T02:06:08.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.770455] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"279570c597f4b22d12169af5","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-279570c597f4b22d12169af5: start","job_status":"start","queue_duration_s":5.414462,"scheduling_latency_s":5.414498} gitlab # [ 291.783239] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"279570c597f4b22d12169af5","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-279570c597f4b22d12169af5: done: 0.022874 sec","job_status":"done","queue_duration_s":5.414462,"scheduling_latency_s":5.414498,"redis_calls":13,"redis_duration_s":0.002224,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022874,"completed_at":"2026-08-27T02:06:08.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.824974] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96587961b1d18ae3ca4b9b59","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96587961b1d18ae3ca4b9b59: start","job_status":"start","queue_duration_s":5.439105,"scheduling_latency_s":5.439141} gitlab # [ 291.837855] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96587961b1d18ae3ca4b9b59","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96587961b1d18ae3ca4b9b59: done: 0.019771 sec","job_status":"done","queue_duration_s":5.439105,"scheduling_latency_s":5.439141,"redis_calls":13,"redis_duration_s":0.0018379999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019771,"completed_at":"2026-08-27T02:06:08.437Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.878632] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a4bb57dfc7547eb9262a512","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a4bb57dfc7547eb9262a512: start","job_status":"start","queue_duration_s":5.460204,"scheduling_latency_s":5.460233} gitlab # [ 291.891652] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a4bb57dfc7547eb9262a512","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a4bb57dfc7547eb9262a512: done: 0.015562 sec","job_status":"done","queue_duration_s":5.460204,"scheduling_latency_s":5.460233,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015562,"completed_at":"2026-08-27T02:06:08.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.932337] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d2ffe63bfb81b0167afd360","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d2ffe63bfb81b0167afd360: start","job_status":"start","queue_duration_s":5.477189,"scheduling_latency_s":5.47722} gitlab # [ 291.945067] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d2ffe63bfb81b0167afd360","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d2ffe63bfb81b0167afd360: done: 0.015576 sec","job_status":"done","queue_duration_s":5.477189,"scheduling_latency_s":5.47722,"redis_calls":13,"redis_duration_s":0.001485,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015576,"completed_at":"2026-08-27T02:06:08.471Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.985384] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf391d05206eac0f06da615f","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf391d05206eac0f06da615f: start","job_status":"start","queue_duration_s":5.494179,"scheduling_latency_s":5.494209} gitlab # [ 291.998192] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf391d05206eac0f06da615f","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf391d05206eac0f06da615f: done: 0.015803 sec","job_status":"done","queue_duration_s":5.494179,"scheduling_latency_s":5.494209,"redis_calls":13,"redis_duration_s":0.001593,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015803,"completed_at":"2026-08-27T02:06:08.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.039794] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87fb17d128b431ed01cd71dd","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87fb17d128b431ed01cd71dd: start","job_status":"start","queue_duration_s":5.511328,"scheduling_latency_s":5.511358} gitlab # [ 292.052827] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87fb17d128b431ed01cd71dd","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87fb17d128b431ed01cd71dd: done: 0.016187 sec","job_status":"done","queue_duration_s":5.511328,"scheduling_latency_s":5.511358,"redis_calls":13,"redis_duration_s":0.0018189999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001256,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016187,"completed_at":"2026-08-27T02:06:08.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.093464] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eebaba21f0ca40e6df30f719","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eebaba21f0ca40e6df30f719: start","job_status":"start","queue_duration_s":5.528896,"scheduling_latency_s":5.528926} gitlab # [ 292.106416] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eebaba21f0ca40e6df30f719","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eebaba21f0ca40e6df30f719: done: 0.015735 sec","job_status":"done","queue_duration_s":5.528896,"scheduling_latency_s":5.528926,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015735,"completed_at":"2026-08-27T02:06:08.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.146840] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e28e9b5440e0138cd6ce7c","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e28e9b5440e0138cd6ce7c: start","job_status":"start","queue_duration_s":5.546053,"scheduling_latency_s":5.546084} gitlab # [ 292.159904] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e28e9b5440e0138cd6ce7c","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e28e9b5440e0138cd6ce7c: done: 0.015618 sec","job_status":"done","queue_duration_s":5.546053,"scheduling_latency_s":5.546084,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015618,"completed_at":"2026-08-27T02:06:08.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.201206] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3352bc804363d09c7e13d4d","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3352bc804363d09c7e13d4d: start","job_status":"start","queue_duration_s":5.563088,"scheduling_latency_s":5.563118} gitlab # [ 292.214872] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3352bc804363d09c7e13d4d","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3352bc804363d09c7e13d4d: done: 0.015592 sec","job_status":"done","queue_duration_s":5.563088,"scheduling_latency_s":5.563118,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015592,"completed_at":"2026-08-27T02:06:08.557Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.256841] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d4a595db888ac2795b0b4a0","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d4a595db888ac2795b0b4a0: start","job_status":"start","queue_duration_s":5.580017,"scheduling_latency_s":5.580044} gitlab # [ 292.269466] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d4a595db888ac2795b0b4a0","created_at":"2026-08-27T02:06:02.356Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-23dcd19f54e945ed","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0f560308253866a7761fa1602c3f70e6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.978Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d4a595db888ac2795b0b4a0: done: 0.01556 sec","job_status":"done","queue_duration_s":5.580017,"scheduling_latency_s":5.580044,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01556,"completed_at":"2026-08-27T02:06:08.574Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.309283] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8cfa973fd9c9c09ad2b00466","created_at":"2026-08-27T02:06:02.977Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06c6816b08ffa97a90f71dcb7ca7580b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.993Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8cfa973fd9c9c09ad2b00466: start","job_status":"start","queue_duration_s":5.582065,"scheduling_latency_s":5.582097} gitlab # [ 292.324174] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.584Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 292.327736] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8cfa973fd9c9c09ad2b00466","created_at":"2026-08-27T02:06:02.977Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06c6816b08ffa97a90f71dcb7ca7580b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:02.993Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8cfa973fd9c9c09ad2b00466: done: 0.014157 sec","job_status":"done","queue_duration_s":5.582065,"scheduling_latency_s":5.582097,"redis_calls":7,"redis_duration_s":0.001016,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014157,"completed_at":"2026-08-27T02:06:08.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.369583] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"60972f948dfcc049afd1bcd4","created_at":"2026-08-27T02:06:03.033Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f115770864f4dd0f5ececbfe01a870fd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.041Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-60972f948dfcc049afd1bcd4: start","job_status":"start","queue_duration_s":5.5502,"scheduling_latency_s":5.55023} gitlab # [ 292.383925] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"60972f948dfcc049afd1bcd4","created_at":"2026-08-27T02:06:03.033Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f115770864f4dd0f5ececbfe01a870fd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.041Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-60972f948dfcc049afd1bcd4: done: 0.035579 sec","job_status":"done","queue_duration_s":5.5502,"scheduling_latency_s":5.55023,"redis_calls":7,"redis_duration_s":0.001121,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.035579,"completed_at":"2026-08-27T02:06:08.626Z","load_balancing_strategy":"primary","db_duration_s":0.018801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.427528] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e6df868752fb92057c0f7b5c","created_at":"2026-08-27T02:06:03.077Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b27a9d44ba9fdc63e27621e00489e6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-27T02:06:03.143Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e6df868752fb92057c0f7b5c: start","job_status":"start","queue_duration_s":5.485462,"scheduling_latency_s":5.485508} gitlab # [ 292.441583] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.636Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 292.444980] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e6df868752fb92057c0f7b5c","created_at":"2026-08-27T02:06:03.077Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b27a9d44ba9fdc63e27621e00489e6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-27T02:06:03.143Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e6df868752fb92057c0f7b5c: done: 0.014039 sec","job_status":"done","queue_duration_s":5.485462,"scheduling_latency_s":5.485508,"redis_calls":7,"redis_duration_s":0.001047,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014039,"completed_at":"2026-08-27T02:06:08.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.485642] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.644Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"8cd136820e3fe1120ece7a27","created_at":"2026-08-27T02:06:03.158Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81e4c0cc5a4e6e7c21e76bfda56186b2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.190Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-8cd136820e3fe1120ece7a27: start","job_status":"start","queue_duration_s":5.453824,"scheduling_latency_s":5.453854} gitlab # [ 292.500442] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"8cd136820e3fe1120ece7a27","created_at":"2026-08-27T02:06:03.158Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81e4c0cc5a4e6e7c21e76bfda56186b2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.190Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-8cd136820e3fe1120ece7a27: done: 0.021442 sec","job_status":"done","queue_duration_s":5.453824,"scheduling_latency_s":5.453854,"redis_calls":12,"redis_duration_s":0.001729,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026462,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.021442,"completed_at":"2026-08-27T02:06:08.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.547165] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e3fa497fcc418a42122ad2ca","created_at":"2026-08-27T02:06:03.220Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"217f09fc760e49945396601d5ffd76ea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.231Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e3fa497fcc418a42122ad2ca: start","job_status":"start","queue_duration_s":5.435734,"scheduling_latency_s":5.435765} gitlab # [ 292.561947] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.946Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e3fa497fcc418a42122ad2ca","created_at":"2026-08-27T02:06:03.220Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"217f09fc760e49945396601d5ffd76ea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.231Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e3fa497fcc418a42122ad2ca: done: 0.279353 sec","job_status":"done","queue_duration_s":5.435734,"scheduling_latency_s":5.435765,"redis_calls":11,"redis_duration_s":0.001889,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000336,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000804,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.058,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.204861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055576,"exclusive_lock_hold_duration_s":0.25987148,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.279353,"completed_at":"2026-08-27T02:06:08.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.086278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.608036] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c1d3c9c7627fc2f5136b5a15","created_at":"2026-08-27T02:06:03.301Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc0659212e0b4f311843d391ad8614c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.314Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c1d3c9c7627fc2f5136b5a15: start","job_status":"start","queue_duration_s":5.633046,"scheduling_latency_s":5.633075} gitlab # [ 292.623072] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.956Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 292.627062] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.961Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c1d3c9c7627fc2f5136b5a15","created_at":"2026-08-27T02:06:03.301Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc0659212e0b4f311843d391ad8614c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.314Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c1d3c9c7627fc2f5136b5a15: done: 0.014256 sec","job_status":"done","queue_duration_s":5.633046,"scheduling_latency_s":5.633075,"redis_calls":7,"redis_duration_s":0.001147,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014256,"completed_at":"2026-08-27T02:06:08.961Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.670721] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.963Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2fe27549623041c98a8e62a4","created_at":"2026-08-27T02:06:03.344Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3431c9730e8dba4e04112f2808ae18c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.357Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2fe27549623041c98a8e62a4: start","job_status":"start","queue_duration_s":5.605504,"scheduling_latency_s":5.605531} gitlab # [ 292.686042] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.971Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 292.689543] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2fe27549623041c98a8e62a4","created_at":"2026-08-27T02:06:03.344Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3431c9730e8dba4e04112f2808ae18c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.357Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2fe27549623041c98a8e62a4: done: 0.014163 sec","job_status":"done","queue_duration_s":5.605504,"scheduling_latency_s":5.605531,"redis_calls":7,"redis_duration_s":0.001003,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014163,"completed_at":"2026-08-27T02:06:08.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.731972] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.979Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"06ab113ac1dd0991104980a7","created_at":"2026-08-27T02:06:03.379Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4980e4cff23c22599a8853c6d3232a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.386Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-06ab113ac1dd0991104980a7: start","job_status":"start","queue_duration_s":5.592467,"scheduling_latency_s":5.592498} gitlab # [ 292.747180] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.988Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 292.750601] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.994Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"06ab113ac1dd0991104980a7","created_at":"2026-08-27T02:06:03.379Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4980e4cff23c22599a8853c6d3232a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.386Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-06ab113ac1dd0991104980a7: done: 0.014948 sec","job_status":"done","queue_duration_s":5.592467,"scheduling_latency_s":5.592498,"redis_calls":7,"redis_duration_s":0.001251,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014948,"completed_at":"2026-08-27T02:06:08.994Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.793364] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:08.995Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c0ecdc38d4daecfcee071fab","created_at":"2026-08-27T02:06:03.445Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2701a9719299286b34007372d6c39ff","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.479Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c0ecdc38d4daecfcee071fab: start","job_status":"start","queue_duration_s":5.515766,"scheduling_latency_s":5.515795} gitlab # [ 292.808550] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c0ecdc38d4daecfcee071fab","created_at":"2026-08-27T02:06:03.445Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2701a9719299286b34007372d6c39ff","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.479Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c0ecdc38d4daecfcee071fab: done: 0.036018 sec","job_status":"done","queue_duration_s":5.515766,"scheduling_latency_s":5.515795,"redis_calls":10,"redis_duration_s":0.001773,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001093,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028146,"duration_s":0.036018,"completed_at":"2026-08-27T02:06:09.031Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.853261] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"5dab0c5b59fbf0e8c3f4006d","created_at":"2026-08-27T02:06:03.541Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ef04d92ec1d5eaddeb49c9347ea62e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.552Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-5dab0c5b59fbf0e8c3f4006d: start","job_status":"start","queue_duration_s":5.481375,"scheduling_latency_s":5.481406} gitlab # [ 292.868434] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.048Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"5dab0c5b59fbf0e8c3f4006d","created_at":"2026-08-27T02:06:03.541Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ef04d92ec1d5eaddeb49c9347ea62e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.552Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-5dab0c5b59fbf0e8c3f4006d: done: 0.014772 sec","job_status":"done","queue_duration_s":5.481375,"scheduling_latency_s":5.481406,"redis_calls":7,"redis_duration_s":0.001094,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000312,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014772,"completed_at":"2026-08-27T02:06:09.048Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.910392] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"aa0877b0d4983ba447c87773","created_at":"2026-08-27T02:06:03.598Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e00e2059f7b09d3a73bcaacc950781c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.608Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-aa0877b0d4983ba447c87773: start","job_status":"start","queue_duration_s":5.441621,"scheduling_latency_s":5.441651} gitlab # [ 292.925527] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"aa0877b0d4983ba447c87773","created_at":"2026-08-27T02:06:03.598Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-b58a562866314d48","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e00e2059f7b09d3a73bcaacc950781c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.608Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-aa0877b0d4983ba447c87773: done: 0.01432 sec","job_status":"done","queue_duration_s":5.441621,"scheduling_latency_s":5.441651,"redis_calls":7,"redis_duration_s":0.0010609999999999999,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000307,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01432,"completed_at":"2026-08-27T02:06:09.064Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.967913] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.065Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"900dd8a8488b1274e56de50a","created_at":"2026-08-27T02:06:03.905Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-25e00c8407c04131","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.917Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-900dd8a8488b1274e56de50a: start","job_status":"start","queue_duration_s":5.147968,"scheduling_latency_s":5.147996} gitlab # [ 292.981041] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.078Z","message":"No existing merge request to be cleaned up."} gitlab # [ 292.982719] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.088Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"900dd8a8488b1274e56de50a","created_at":"2026-08-27T02:06:03.905Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-25e00c8407c04131","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.917Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-900dd8a8488b1274e56de50a: done: 0.022738 sec","job_status":"done","queue_duration_s":5.147968,"scheduling_latency_s":5.147996,"redis_calls":10,"redis_duration_s":0.001496,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022738,"completed_at":"2026-08-27T02:06:09.088Z","load_balancing_strategy":"primary","db_duration_s":0.003384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 293.022911] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.090Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8000ea93c9d6c022dbfb7e8","created_at":"2026-08-27T02:06:03.905Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-25e00c8407c04131","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.917Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8000ea93c9d6c022dbfb7e8: start","job_status":"start","queue_duration_s":5.172115,"scheduling_latency_s":5.172144} gitlab # [ 293.035914] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.102Z","message":"No existing merge request to be cleaned up."} gitlab # [ 293.038691] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.112Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8000ea93c9d6c022dbfb7e8","created_at":"2026-08-27T02:06:03.905Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-25e00c8407c04131","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.917Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8000ea93c9d6c022dbfb7e8: done: 0.022515 sec","job_status":"done","queue_duration_s":5.172115,"scheduling_latency_s":5.172144,"redis_calls":10,"redis_duration_s":0.001941,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022515,"completed_at":"2026-08-27T02:06:09.112Z","load_balancing_strategy":"primary","db_duration_s":0.002752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 293.078993] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.113Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"36cdcb2a0744d712f4071383","created_at":"2026-08-27T02:06:03.905Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-25e00c8407c04131","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.917Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-36cdcb2a0744d712f4071383: start","job_status":"start","queue_duration_s":5.196022,"scheduling_latency_s":5.196052} gitlab # [ 293.091971] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.126Z","message":"No existing merge request to be cleaned up."} gitlab # [ 293.093847] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.135Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"36cdcb2a0744d712f4071383","created_at":"2026-08-27T02:06:03.905Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-25e00c8407c04131","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.917Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-36cdcb2a0744d712f4071383: done: 0.021902 sec","job_status":"done","queue_duration_s":5.196022,"scheduling_latency_s":5.196052,"redis_calls":10,"redis_duration_s":0.001499,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021902,"completed_at":"2026-08-27T02:06:09.135Z","load_balancing_strategy":"primary","db_duration_s":0.002586,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 293.133920] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.137Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5f205aa413bf00e9f19341f5","created_at":"2026-08-27T02:06:03.905Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-25e00c8407c04131","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.917Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5f205aa413bf00e9f19341f5: start","job_status":"start","queue_duration_s":5.219335,"scheduling_latency_s":5.219366} gitlab # [ 293.146814] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.149Z","message":"No existing merge request to be cleaned up."} gitlab # [ 293.148551] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:09.158Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5f205aa413bf00e9f19341f5","created_at":"2026-08-27T02:06:03.905Z","trace_propagation_headers":{"sentry-trace":"b54e20c2e3bc4ad4b89d68d59bf8fe55-25e00c8407c04131","baggage":"sentry-trace_id=b54e20c2e3bc4ad4b89d68d59bf8fe55,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d62c0d0808a40db7fcb95f8c536615c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:06:03.917Z","job_size_bytes":2,"pid":2073,"sidekiq_tid":"m7ox","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5f205aa413bf00e9f19341f5: done: 0.021759 sec","job_status":"done","queue_duration_s":5.219335,"scheduling_latency_s":5.219366,"redis_calls":10,"redis_duration_s":0.001765,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001106,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021759,"completed_at":"2026-08-27T02:06:09.158Z","load_balancing_strategy":"primary","db_duration_s":0.002601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.146176] redis[794]: 10 changes in 300 seconds. Saving... gitlab # [ 309.152864] redis[794]: Background saving started by pid 3345 gitlab # [ 309.156848] redis[3345]: BGSAVE done, 458 keys saved, 0 keys skipped, 118558 bytes written. gitlab # [ 309.162033] redis[3345]: DB saved on disk gitlab # [ 309.164298] redis[3345]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB gitlab # [ 309.247597] redis[794]: Background saving terminated with success gitlab # [ 310.547643] postgres[865]: [865] LOG: checkpoint starting: time gitlab # [ 311.788926] gitaly[906]: time="2026-08-27T02:06:28.658Z" level=info msg="burden monitor stats" pid=906 gitlab # [ 327.884819] postgres[3405]: [3405] ERROR: relation "table_without_models" does not exist gitlab # [ 327.885441] postgres[3405]: [3405] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 333.723329] runuser[3060]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 333.724727] setup-gitlab-start[3030]: + systemctl restart gitlab-sidekiq.service gitlab # [ 333.765645] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 333.772111] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:50.635Z","message":"Shutting down"} gitlab # [ 333.776404] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:50.636Z","message":"Scheduler exiting..."} gitlab # [ 333.780790] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:50.637Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 333.786016] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:50.638Z","message":"Scheduler exiting..."} gitlab # [ 334.268231] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:51.139Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 337.773120] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:54.643Z","message":"Bye!"} gitlab # [ 337.814737] sidekiq-cluster[2073]: {"severity":"INFO","time":"2026-08-27T02:06:54.686Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2073,"worker_id":"sidekiq_0","memwd_rss_bytes":927780864,"retry":0} gitlab # [ 339.020611] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 339.021034] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 339.021260] systemd[1]: gitlab-sidekiq.service: Consumed 1min 57.937s CPU time over 3min 29.759s wall clock time, 936.6M memory peak, 4.5M read from disk. gitlab # [ 339.035198] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 339.041532] setup-gitlab-start[3030]: + mkdir -p /var/lib/secrets gitlab # [ 339.047134] setup-gitlab-start[3030]: + echo nixbot-test-token-of-20-chars gitlab # [ 339.050191] setup-gitlab-start[3030]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 339.053272] setup-gitlab-start[3030]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 340.851559] gitaly[906]: time="2026-08-27T02:06:57.720Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:07.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:57.719" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.882576] gitaly[906]: time="2026-08-27T02:06:57.721Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:07.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:57.719" grpc.time_ms=1.383 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.287417] gitaly[906]: time="2026-08-27T02:06:58.158Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.158" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.349121] gitaly[906]: time="2026-08-27T02:06:58.159Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.158" grpc.time_ms=1.164 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.466482] gitaly[906]: time="2026-08-27T02:06:58.337Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M diskcache=5938fb4a-e701-45b8-8c76-552c7f4c0cfb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-27T02:07:08.093" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.244" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.492929] gitaly[906]: time="2026-08-27T02:06:58.338Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=50 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=62 command.spawn_token_fork_ms=12 command.system_time_ms=36 command.user_time_ms=14 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-27T02:07:08.093" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.244" grpc.time_ms=94.023 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.517178] gitaly[906]: time="2026-08-27T02:06:58.381Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.229" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.380" pid=906 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 341.534438] gitaly[906]: time="2026-08-27T02:06:58.382Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.229" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.380" grpc.time_ms=1.522 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.550559] gitaly[906]: time="2026-08-27T02:06:58.417Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=8 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-27T02:07:08.242" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.393" grpc.time_ms=23.888 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.708937] gitaly[906]: time="2026-08-27T02:06:58.580Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.579" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.741265] gitaly[906]: time="2026-08-27T02:06:58.581Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.579" grpc.time_ms=1.036 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.790037] gitaly[906]: time="2026-08-27T02:06:58.661Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M diskcache=fa56a1fa-3227-480d-a828-3c478bd0f9f9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-27T02:07:08.016" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.586" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.819161] gitaly[906]: time="2026-08-27T02:06:58.662Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=46 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1440 command.oublock=32 command.real_time_ms=50 command.spawn_token_fork_ms=7 command.system_time_ms=35 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-27T02:07:08.016" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.586" grpc.time_ms=75.041 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.844297] gitaly[906]: time="2026-08-27T02:06:58.685Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.111" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.682" pid=906 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 341.863494] gitaly[906]: time="2026-08-27T02:06:58.686Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.111" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.682" grpc.time_ms=3.142 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.881626] gitaly[906]: time="2026-08-27T02:06:58.719Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=3 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-27T02:07:08.124" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.695" grpc.time_ms=23.423 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.925229] gitaly[906]: time="2026-08-27T02:06:58.796Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:07:08.084" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.795" grpc.time_ms=0.709 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.949858] gitaly[906]: time="2026-08-27T02:06:58.821Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-27T02:07:08.108" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:58.820" grpc.time_ms=0.736 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 343.013445] gitaly[906]: time="2026-08-27T02:06:59.883Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGSEGN1Q8B0BWNHVMHX7M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-27T02:07:09.155" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:06:59.866" grpc.time_ms=17.616 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 343.472916] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:00 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 4387 gitlab # [ 343.482311] setup-gitlab-start[3030]: + mkdir -p /tmp/test-flake gitlab # [ 343.486270] setup-gitlab-start[3030]: + cd /tmp/test-flake gitlab # [ 343.489660] setup-gitlab-start[3030]: + git init -b master gitlab # [ 343.518568] setup-gitlab-start[3527]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 343.524628] setup-gitlab-start[3030]: + git config user.name test gitlab # [ 343.531013] setup-gitlab-start[3030]: + git config user.email test@example.com gitlab # [ 343.538313] setup-gitlab-start[3030]: + cat gitlab # [ 343.544485] setup-gitlab-start[3030]: + git add flake.nix gitlab # [ 343.553099] setup-gitlab-start[3030]: + git commit -m 'initial commit' gitlab # [ 343.581643] setup-gitlab-start[3536]: [master (root-commit) 9e7da77] initial commit gitlab # [ 343.586037] setup-gitlab-start[3536]: 1 file changed, 10 insertions(+) gitlab # [ 343.589011] setup-gitlab-start[3536]: create mode 100644 flake.nix gitlab # [ 343.591679] setup-gitlab-start[3030]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 343.597133] setup-gitlab-start[3030]: + git push -u origin master gitlab # [ 343.730123] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:00 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 89 gitlab # [ 343.956456] gitaly[906]: time="2026-08-27T02:07:00.827Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGXZVRZ45FWMZ69YENBZJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:00.810" grpc.time_ms=16.666 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 343.996815] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:00 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 224 gitlab # [ 344.363708] sidekiq-cluster[3472]: {"severity":"INFO","time":"2026-08-27T02:07:01.192Z","message":"Starting cluster with 1 processes"} gitlab # [ 344.700362] gitaly[906]: time="2026-08-27T02:07:01.570Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-27T02:07:11.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:07:01.553" grpc.time_ms=16.904 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 344.756920] gitaly[906]: time="2026-08-27T02:07:01.628Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-08-27T02:07:31.015" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-08-27T02:07:01.616" grpc.time_ms=11.815 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 344.837507] gitaly[906]: time="2026-08-27T02:07:01.707Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M10FGY7BS202X1T11VQVMM02 duration_ms=644ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 344.893222] gitaly[906]: time="2026-08-27T02:07:01.764Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M10FGY7BS202X1T11VQVMM02 duration_ms=55ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 344.902738] gitaly[906]: time="2026-08-27T02:07:01.765Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:01.060" grpc.time_ms=704.465 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 344.950518] gitaly[906]: time="2026-08-27T02:07:01.822Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:01.820" grpc.time_ms=1.49 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 345.012986] gitaly[906]: time="2026-08-27T02:07:01.884Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:01.882" grpc.time_ms=1.889 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 345.067388] gitaly[906]: time="2026-08-27T02:07:01.938Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:01.937" grpc.time_ms=0.951 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 345.247973] gitaly[906]: time="2026-08-27T02:07:02.119Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-27T02:07:12.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:07:02.101" grpc.time_ms=17.956 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.323440] gitaly[906]: time="2026-08-27T02:07:02.194Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1436 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=3 command.system_time_ms=17 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-08-27T02:07:12.006" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:02.157" grpc.time_ms=37.301 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 345.363914] gitaly[906]: time="2026-08-27T02:07:02.235Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M10FGY7BS202X1T11VQVMM02 duration_ms=252ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 345.370807] gitaly[906]: time="2026-08-27T02:07:02.236Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:01.980" grpc.time_ms=256.537 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 345.394446] gitaly[906]: time="2026-08-27T02:07:02.252Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 diskcache=dca145ab-0182-4dba-87ff-38c36064dc31 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:00.943" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 345.415061] setup-gitlab-start[3540]: To http://gitlab/root/test-flake.git gitlab # [ 345.416746] setup-gitlab-start[3540]: * [new branch] master -> master gitlab # [ 345.418277] setup-gitlab-start[3540]: branch 'master' set up to track 'origin/master'. gitlab # [ 345.420568] gitlab-workhorse[824]: time="2026-08-27T02:07:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 345.423450] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:02 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1409 gitlab # [ 345.426940] systemd[1]: Finished setup-gitlab.service. gitlab # [ 345.428115] gitaly[906]: time="2026-08-27T02:07:02.252Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=343 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13852 command.oublock=32 command.real_time_ms=1297 command.spawn_token_fork_ms=2 command.system_time_ms=181 command.user_time_ms=162 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:00.943" grpc.time_ms=1308.588 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 345.446537] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 346.23 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 346.805786] nixbot[3632]: {"ts": "2026-08-27T02:07:03.676Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 346.833348] nixbot[3632]: {"ts": "2026-08-27T02:07:03.704Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 346.854422] nixbot[3632]: {"ts": "2026-08-27T02:07:03.725Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 346.861479] nixbot[3632]: {"ts": "2026-08-27T02:07:03.730Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 346.868414] nixbot[3632]: {"ts": "2026-08-27T02:07:03.734Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 346.875840] nixbot[3632]: {"ts": "2026-08-27T02:07:03.741Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 346.882554] nixbot[3632]: {"ts": "2026-08-27T02:07:03.745Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 346.889695] nixbot[3632]: {"ts": "2026-08-27T02:07:03.747Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 346.896293] nixbot[3632]: {"ts": "2026-08-27T02:07:03.753Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 346.902990] nixbot[3632]: {"ts": "2026-08-27T02:07:03.755Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 346.910068] nixbot[3632]: {"ts": "2026-08-27T02:07:03.760Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 346.916369] nixbot[3632]: {"ts": "2026-08-27T02:07:03.766Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 346.922449] nixbot[3632]: {"ts": "2026-08-27T02:07:03.769Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 346.928901] nixbot[3632]: {"ts": "2026-08-27T02:07:03.771Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 346.934803] nixbot[3632]: {"ts": "2026-08-27T02:07:03.775Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 346.940182] nixbot[3632]: {"ts": "2026-08-27T02:07:03.777Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 346.944674] nixbot[3632]: {"ts": "2026-08-27T02:07:03.779Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 346.950249] nixbot[3632]: {"ts": "2026-08-27T02:07:03.780Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 346.957208] nixbot[3632]: {"ts": "2026-08-27T02:07:03.784Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 346.964397] nixbot[3632]: {"ts": "2026-08-27T02:07:03.785Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 346.966593] nixbot[3632]: {"ts": "2026-08-27T02:07:03.787Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 346.968797] nixbot[3632]: {"ts": "2026-08-27T02:07:03.792Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 346.971206] nixbot[3632]: {"ts": "2026-08-27T02:07:03.793Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 346.973831] nixbot[3632]: {"ts": "2026-08-27T02:07:03.795Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 346.976234] nixbot[3632]: {"ts": "2026-08-27T02:07:03.796Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 347.324985] nixbot[3632]: INFO: Started server process [3632] gitlab # [ 347.327795] nixbot[3632]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 347.348266] nixbot[3632]: INFO: ::ffff:127.0.0.1:50730 - "GET /health HTTP/1.1" 200 OK gitlab # [ 347.358600] systemd[1]: Started nixbot CI service. gitlab # [ 347.361396] systemd[1]: Reached target Multi-User System. gitlab # [ 347.364434] systemd[1]: Startup finished in 724ms (kernel) + 4.193s (initrd) + 5min 42.446s (userspace) = 5min 47.364s. gitlab: (finished: waiting for unit nixbot.service, in 2.15 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 347.995700] nixbot[3632]: INFO: ::ffff:127.0.0.1:50738 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.06 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 348.016726] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:04 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 667 gitlab # [ 348.019323] nixbot[3632]: {"ts": "2026-08-27T02:07:04.890Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 348.066810] nixbot[3632]: {"ts": "2026-08-27T02:07:04.938Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 348.180553] nixbot[3632]: {"ts": "2026-08-27T02:07:05.051Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 348.188681] nixbot[3632]: {"ts": "2026-08-27T02:07:05.053Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 348.193662] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 102 gitlab # [ 348.326820] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 269 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.33 seconds) gitlab # [ 348.390212] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 202 gitlab # [ 348.394742] nixbot[3632]: {"ts": "2026-08-27T02:07:05.263Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 348.684576] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:05 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 279 gitlab # [ 348.687200] nixbot[3632]: {"ts": "2026-08-27T02:07:05.557Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 348.807893] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 101 gitlab # [ 348.815188] nixbot[3632]: {"ts": "2026-08-27T02:07:05.680Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 348.978429] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:05 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 164 gitlab # [ 348.980800] nixbot[3632]: {"ts": "2026-08-27T02:07:05.852Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 349.093124] gitaly[906]: time="2026-08-27T02:07:05.963Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH349WEXDM4BCS9H3MNZY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-08-27T02:07:35.334" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:05.935" grpc.time_ms=27.221 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 349.138283] gitaly[906]: time="2026-08-27T02:07:05.988Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH349WEXDM4BCS9H3MNZY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:07:35.385" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:07:05.986" grpc.time_ms=1.431 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 349.162939] gitaly[906]: time="2026-08-27T02:07:06.010Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH349WEXDM4BCS9H3MNZY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-08-27T02:07:15.002" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:07:05.993" grpc.time_ms=16.882 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 349.183832] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 184 gitlab # [ 349.187529] nixbot[3632]: {"ts": "2026-08-27T02:07:06.050Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 349.311167] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 127 gitlab # [ 349.314127] nixbot[3632]: {"ts": "2026-08-27T02:07:06.184Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 349.320731] nixbot[3632]: {"ts": "2026-08-27T02:07:06.192Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 349.474693] gitaly[906]: time="2026-08-27T02:07:06.346Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=8 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FH3GNK7XR0P3HVMC30945 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.334" grpc.time_ms=10.935 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 349.495576] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 85 gitlab # [ 349.541676] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 178 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.21 seconds) (finished: subtest: project discovered and webhook registered, in 1.54 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 349.691847] gitaly[906]: time="2026-08-27T02:07:06.563Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FH3KCBKD7YXNFE74R1X4M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.548" pid=906 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 349.715656] gitaly[906]: time="2026-08-27T02:07:06.568Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH3KCBKD7YXNFE74R1X4M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.548" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 349.739161] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 66 gitlab # [ 349.742409] gitlab-workhorse[824]: time="2026-08-27T02:07:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 349.745669] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 220 gitlab # [ 349.748408] gitaly[906]: time="2026-08-27T02:07:06.569Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH3KCBKD7YXNFE74R1X4M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.548" grpc.time_ms=19.636 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 349.794449] gitaly[906]: time="2026-08-27T02:07:06.665Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FH3TFC6KYPZN3XBJDAA22 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.648" pid=906 remote_ip="2001:db8:1::1" request_sha=4493fda11945d82aacb201f8474c9a57666c031e response_bytes=223 span.kind=server system=grpc gitlab # [ 349.810704] gitaly[906]: time="2026-08-27T02:07:06.665Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH3TFC6KYPZN3XBJDAA22 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.648" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 349.832400] gitlab-workhorse[824]: time="2026-08-27T02:07:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 349.834515] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 90 gitlab # [ 349.836609] gitaly[906]: time="2026-08-27T02:07:06.666Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH3TFC6KYPZN3XBJDAA22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.648" grpc.time_ms=17.702 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 349.852613] gitaly[906]: time="2026-08-27T02:07:06.716Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M10FH3TCDMYQAHJPR701KRNB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.703" grpc.time_ms=12.895 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # Enumerating objects: 5, done. gitlab # [ 349.867944] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 144 gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 349.919573] gitaly[906]: time="2026-08-27T02:07:06.791Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FH3YGF0B6045X4H7YBPT2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.776" grpc.time_ms=14.595 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 349.935472] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 86 gitlab # [ 350.017172] gitaly[906]: time="2026-08-27T02:07:06.888Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FH419EY547ANFJW98Z4FN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.880" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 350.028140] gitaly[906]: time="2026-08-27T02:07:06.888Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH419EY547ANFJW98Z4FN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.880" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 350.042843] gitlab-workhorse[824]: time="2026-08-27T02:07:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 350.045052] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 96 gitlab # [ 350.047255] gitaly[906]: time="2026-08-27T02:07:06.889Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH419EY547ANFJW98Z4FN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.880" grpc.time_ms=9.049 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 350.137908] gitaly[906]: time="2026-08-27T02:07:07.008Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M10FH45KPT8HSXM2JNXX0SYQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.999" grpc.time_ms=8.449 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 350.170427] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 80 gitlab # [ 350.196126] gitaly[906]: time="2026-08-27T02:07:07.067Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-08-27T02:07:37.052" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-08-27T02:07:07.052" grpc.time_ms=14.982 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 350.229981] gitaly[906]: time="2026-08-27T02:07:07.101Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M10FH402T6CB0A36PR2ERK51 duration_ms=180ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 350.599404] gitaly[906]: time="2026-08-27T02:07:07.470Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FH486WZFT5BHG190V8XR4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:07.440" pid=906 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 350.624539] gitaly[906]: time="2026-08-27T02:07:07.471Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=3 command.system_time_ms=19 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH486WZFT5BHG190V8XR4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:07.440" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 350.649865] gitlab-workhorse[824]: time="2026-08-27T02:07:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 350.652946] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 458 gitlab # [ 350.655755] gitaly[906]: time="2026-08-27T02:07:07.482Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=3 command.system_time_ms=19 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH486WZFT5BHG190V8XR4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:07.440" grpc.time_ms=31.099 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 350.676458] gitaly[906]: time="2026-08-27T02:07:07.483Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M10FH402T6CB0A36PR2ERK51 duration_ms=381ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 350.681539] gitaly[906]: time="2026-08-27T02:07:07.484Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:06.919" grpc.time_ms=563.837 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 350.695107] gitaly[906]: time="2026-08-27T02:07:07.561Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:07.559" grpc.time_ms=0.905 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 350.730959] gitaly[906]: time="2026-08-27T02:07:07.602Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:07.600" grpc.time_ms=1.707 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 350.769195] gitaly[906]: time="2026-08-27T02:07:07.640Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:07.639" grpc.time_ms=0.739 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 350.806318] gitaly[906]: time="2026-08-27T02:07:07.677Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:07:37.076" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:07:07.676" grpc.time_ms=1.107 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 350.870564] gitaly[906]: time="2026-08-27T02:07:07.742Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-08-27T02:07:37.131" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:07.731" grpc.time_ms=10.266 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 350.904898] gitaly[906]: time="2026-08-27T02:07:07.776Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M10FH402T6CB0A36PR2ERK51 duration_ms=91ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 350.909622] gitaly[906]: time="2026-08-27T02:07:07.777Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:07.684" grpc.time_ms=92.404 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 350.924908] gitaly[906]: time="2026-08-27T02:07:07.790Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:07:17.230" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:07.780" grpc.time_ms=9.995 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 350.943867] gitaly[906]: time="2026-08-27T02:07:07.802Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 diskcache=e82d8470-2972-4fb2-8532-c60e21971bbe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.844" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # 9e7da77..029b33b [ 350.956392] gitlab-workhorse[824]: time="2026-08-27T02:07:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # master -> master[ 350.958712] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:07 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1049 gitlab # gitlab # [ 350.961106] gitaly[906]: time="2026-08-27T02:07:07.802Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=194 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13193 command.oublock=32 command.real_time_ms=939 command.spawn_token_fork_ms=1 command.system_time_ms=103 command.user_time_ms=91 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:06.844" grpc.time_ms=958.052 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 350.976756] gitaly[906]: time="2026-08-27T02:07:07.825Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:07:17.262" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:07.812" grpc.time_ms=13.041 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.44 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 351.087147] nixbot[3632]: INFO: ::ffff:127.0.0.1:34012 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.09 seconds) gitlab # [ 351.092701] gitaly[906]: time="2026-08-27T02:07:07.964Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-27T02:07:17.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:07:07.963" grpc.time_ms=0.467 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.113107] gitaly[906]: time="2026-08-27T02:07:07.982Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-27T02:07:17.014" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:07.974" grpc.time_ms=7.937 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.151675] gitaly[906]: time="2026-08-27T02:07:08.022Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:08.022" grpc.time_ms=0.513 pid=906 span.kind=server system=grpc gitlab # [ 351.179953] gitaly[906]: time="2026-08-27T02:07:08.051Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:07:08.051" grpc.time_ms=0.412 pid=906 span.kind=server system=grpc gitlab # [ 351.196952] gitaly[906]: time="2026-08-27T02:07:08.058Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 diskcache=f7edfffb-0aa7-4f29-afdf-ce56c9f7eca6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-27T02:07:17.025" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:07:07.986" pid=906 ref=refs/keep-around/9e7da772ac2fa6f1071cffc107848b72a7804eb7 remote_ip="2001:db8:1::1" revision=9e7da772ac2fa6f1071cffc107848b72a7804eb7 span.kind=server system=grpc username=root gitlab # [ 351.213490] gitaly[906]: time="2026-08-27T02:07:08.058Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5165 command.oublock=8 command.real_time_ms=66 command.spawn_token_fork_ms=0 command.system_time_ms=25 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-27T02:07:17.025" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:07:07.986" grpc.time_ms=72.182 pid=906 ref=refs/keep-around/9e7da772ac2fa6f1071cffc107848b72a7804eb7 remote_ip="2001:db8:1::1" revision=9e7da772ac2fa6f1071cffc107848b72a7804eb7 span.kind=server system=grpc username=root gitlab # [ 351.231707] gitaly[906]: time="2026-08-27T02:07:08.062Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:07:38.062" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:07:08.062" grpc.time_ms=0.36 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.423538] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 746 gitlab # [ 351.432445] nixbot[3632]: {"ts": "2026-08-27T02:07:08.297Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 \"HTTP/1.1 201 Created\""} gitlab # [ 351.443899] nixbot[3632]: {"ts": "2026-08-27T02:07:08.315Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-9e7da772ac2f-0"} gitlab # [ 351.706215] nixbot[3632]: {"ts": "2026-08-27T02:07:08.577Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-9e7da772ac2f-0", "argv0": "bwrap"} gitlab # [ 351.896208] systemd[1]: Started Nix Daemon. gitlab # [ 352.012898] nix-daemon[3863]: accepted connection from pid 3858, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 352.149766] nixbot[3632]: INFO: ::ffff:127.0.0.1:34028 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.07 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 353.208481] nixbot[3632]: INFO: ::ffff:127.0.0.1:34040 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 354.264182] nixbot[3632]: INFO: ::ffff:127.0.0.1:34048 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 355.318719] nixbot[3632]: INFO: ::ffff:127.0.0.1:34062 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 356.377563] nixbot[3632]: INFO: ::ffff:127.0.0.1:34072 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 356.486848] nix-daemon[3863]: accepted connection from pid 3891, user nixbot gitlab # [ 356.507542] nix-daemon[3863]: accepted connection from pid 3858, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 357.439813] nixbot[3632]: INFO: ::ffff:127.0.0.1:34074 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 358.497619] nixbot[3632]: INFO: ::ffff:127.0.0.1:34084 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 359.552720] nixbot[3632]: INFO: ::ffff:127.0.0.1:34086 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 360.608391] nixbot[3632]: INFO: ::ffff:127.0.0.1:34090 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 360.937712] gitaly[906]: time="2026-08-27T02:07:17.808Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FHENRCZPTDQ01MT5T4034 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:07:47.205" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:07:17.806" grpc.time_ms=1.617 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 361.150980] gitaly[906]: time="2026-08-27T02:07:18.022Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FHENRCZPTDQ01MT5T4034 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:07:28.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:18.004" grpc.time_ms=17.735 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 361.270517] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 451 gitlab # [ 361.278654] nixbot[3632]: {"ts": "2026-08-27T02:07:18.143Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 \"HTTP/1.1 201 Created\""} gitlab # [ 361.375314] gitaly[906]: time="2026-08-27T02:07:18.246Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FHF48CWM95S5ZMV8RNV9K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:07:48.244" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:07:18.245" grpc.time_ms=1.019 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 361.438546] gitaly[906]: time="2026-08-27T02:07:18.310Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FHF48CWM95S5ZMV8RNV9K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:07:28.290" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:18.291" grpc.time_ms=18.332 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 361.519875] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 238 gitlab # [ 361.529306] nixbot[3632]: {"ts": "2026-08-27T02:07:18.393Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 361.616315] gitaly[906]: time="2026-08-27T02:07:18.487Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FHFCEBE6GWQVXC8CHNYE6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:07:48.485" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:07:18.486" grpc.time_ms=1.105 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 361.666793] nixbot[3632]: INFO: ::ffff:127.0.0.1:59714 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 361.709313] gitaly[906]: time="2026-08-27T02:07:18.580Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FHFCEBE6GWQVXC8CHNYE6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:07:28.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:07:18.564" grpc.time_ms=16.076 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 361.786046] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 242 gitlab # [ 361.794433] nixbot[3632]: {"ts": "2026-08-27T02:07:18.658Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 \"HTTP/1.1 201 Created\""} gitlab # [ 362.133618] gitaly[906]: time="2026-08-27T02:07:19.004Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M10FHFV3HEZG9JJ90PFWDNXZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:18.985" grpc.time_ms=18.966 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 362.162868] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:19 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 121 gitlab # [ 362.273261] gitaly[906]: time="2026-08-27T02:07:19.144Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FHFZ0GFQ7NG26GGN2Q0BJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:19.125" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 362.303455] gitaly[906]: time="2026-08-27T02:07:19.145Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FHFZ0GFQ7NG26GGN2Q0BJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:19.125" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 362.334013] gitlab-workhorse[824]: time="2026-08-27T02:07:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 362.338111] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 138 gitlab # [ 362.342140] gitaly[906]: time="2026-08-27T02:07:19.145Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FHFZ0GFQ7NG26GGN2Q0BJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:19.125" grpc.time_ms=19.917 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 362.418149] gitaly[906]: time="2026-08-27T02:07:19.289Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FHG3E04HG8YTNZYV7WQ8R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:19.255" pid=906 remote_ip="2001:db8:1::1" request_sha=a6fe355fac12b195b09d7e46d0e3d146ee9a5411 response_bytes=337 span.kind=server system=grpc gitlab # [ 362.436821] gitaly[906]: time="2026-08-27T02:07:19.290Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1131 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=2 command.system_time_ms=25 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M10FHG3E04HG8YTNZYV7WQ8R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:19.255" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 362.460390] gitlab-workhorse[824]: time="2026-08-27T02:07:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 362.463556] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:07:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 141 gitlab # [ 362.466826] gitaly[906]: time="2026-08-27T02:07:19.290Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1131 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=2 command.system_time_ms=25 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M10FHG3E04HG8YTNZYV7WQ8R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:07:19.255" grpc.time_ms=34.541 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 362.731246] nixbot[3632]: INFO: ::ffff:127.0.0.1:59716 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 363.785154] nixbot[3632]: INFO: ::ffff:127.0.0.1:59718 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 364.841968] nixbot[3632]: INFO: ::ffff:127.0.0.1:59722 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 365.897689] nixbot[3632]: INFO: ::ffff:127.0.0.1:59728 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 366.956507] nixbot[3632]: INFO: ::ffff:127.0.0.1:59742 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 368.014138] nixbot[3632]: INFO: ::ffff:127.0.0.1:59758 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 369.070897] nixbot[3632]: INFO: ::ffff:127.0.0.1:59770 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 370.126847] nixbot[3632]: INFO: ::ffff:127.0.0.1:59786 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 371.184049] nixbot[3632]: INFO: ::ffff:127.0.0.1:52566 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 372.240316] nixbot[3632]: INFO: ::ffff:127.0.0.1:52572 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 373.296392] nixbot[3632]: INFO: ::ffff:127.0.0.1:52576 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 374.352741] nixbot[3632]: INFO: ::ffff:127.0.0.1:52590 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 375.408559] nixbot[3632]: INFO: ::ffff:127.0.0.1:52604 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 376.466731] nixbot[3632]: INFO: ::ffff:127.0.0.1:52606 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 377.523926] nixbot[3632]: INFO: ::ffff:127.0.0.1:52612 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 378.580577] nixbot[3632]: INFO: ::ffff:127.0.0.1:52614 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 379.635520] nixbot[3632]: INFO: ::ffff:127.0.0.1:52620 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 380.690776] nixbot[3632]: INFO: ::ffff:127.0.0.1:52634 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 381.748498] nixbot[3632]: INFO: ::ffff:127.0.0.1:46390 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 382.804233] nixbot[3632]: INFO: ::ffff:127.0.0.1:46394 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 383.861388] nixbot[3632]: INFO: ::ffff:127.0.0.1:46396 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 384.918936] nixbot[3632]: INFO: ::ffff:127.0.0.1:46400 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 385.495446] sidekiq-cluster[3569]: 2026-08-27T02:07:42.365Z pid=3569 tid=ymx INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3569, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426196992} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 385.975536] nixbot[3632]: INFO: ::ffff:127.0.0.1:46408 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 385.999877] sidekiq-cluster[3569]: W, [2026-08-27T02:07:42.871437 #3569] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 386.069460] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:07:42.940Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 387.030805] nixbot[3632]: INFO: ::ffff:127.0.0.1:46416 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 388.090294] nixbot[3632]: INFO: ::ffff:127.0.0.1:46422 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 389.078402] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:07:45.948Z","message":"GitLab reliable fetch activated!"} gitlab # [ 389.083916] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:07:45.949Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 389.146586] nixbot[3632]: INFO: ::ffff:127.0.0.1:46428 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 390.199585] nixbot[3632]: INFO: ::ffff:127.0.0.1:46434 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 391.256848] nixbot[3632]: INFO: ::ffff:127.0.0.1:49524 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 392.314969] nixbot[3632]: INFO: ::ffff:127.0.0.1:49528 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 393.369545] nixbot[3632]: INFO: ::ffff:127.0.0.1:49544 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 394.426363] nixbot[3632]: INFO: ::ffff:127.0.0.1:49556 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 395.482611] nixbot[3632]: INFO: ::ffff:127.0.0.1:49568 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 396.539438] nixbot[3632]: INFO: ::ffff:127.0.0.1:49578 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 397.595976] nixbot[3632]: INFO: ::ffff:127.0.0.1:49592 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 398.652356] nixbot[3632]: INFO: ::ffff:127.0.0.1:49608 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 399.708570] nixbot[3632]: INFO: ::ffff:127.0.0.1:49618 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 400.764920] nixbot[3632]: INFO: ::ffff:127.0.0.1:49628 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 401.821503] nixbot[3632]: INFO: ::ffff:127.0.0.1:40082 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 402.876917] nixbot[3632]: INFO: ::ffff:127.0.0.1:40086 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 403.933917] nixbot[3632]: INFO: ::ffff:127.0.0.1:40094 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 404.991943] nixbot[3632]: INFO: ::ffff:127.0.0.1:40098 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 406.049490] nixbot[3632]: INFO: ::ffff:127.0.0.1:40108 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 407.107372] nixbot[3632]: INFO: ::ffff:127.0.0.1:40120 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 408.164675] nixbot[3632]: INFO: ::ffff:127.0.0.1:40134 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 409.224428] nixbot[3632]: INFO: ::ffff:127.0.0.1:40150 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 410.282705] nixbot[3632]: INFO: ::ffff:127.0.0.1:40160 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 411.341996] nixbot[3632]: INFO: ::ffff:127.0.0.1:44594 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 412.399183] nixbot[3632]: INFO: ::ffff:127.0.0.1:44596 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 413.455503] nixbot[3632]: INFO: ::ffff:127.0.0.1:44608 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 414.514082] nixbot[3632]: INFO: ::ffff:127.0.0.1:44614 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 415.571818] nixbot[3632]: INFO: ::ffff:127.0.0.1:44620 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 416.629848] nixbot[3632]: INFO: ::ffff:127.0.0.1:44622 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 417.687283] nixbot[3632]: INFO: ::ffff:127.0.0.1:44638 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 418.744908] nixbot[3632]: INFO: ::ffff:127.0.0.1:44648 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 419.802700] nixbot[3632]: INFO: ::ffff:127.0.0.1:44650 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 420.859645] nixbot[3632]: INFO: ::ffff:127.0.0.1:58394 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 421.919342] nixbot[3632]: INFO: ::ffff:127.0.0.1:58406 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 422.975502] nixbot[3632]: INFO: ::ffff:127.0.0.1:58408 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.05 seconds) gitlab # [ 424.029603] nixbot[3632]: INFO: ::ffff:127.0.0.1:58412 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 425.085236] nixbot[3632]: INFO: ::ffff:127.0.0.1:58428 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 426.144179] nixbot[3632]: INFO: ::ffff:127.0.0.1:58442 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 426.653159] postgres[4162]: [4162] ERROR: relation "table_without_models" does not exist gitlab # [ 426.656852] postgres[4162]: [4162] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 427.200433] nixbot[3632]: INFO: ::ffff:127.0.0.1:58446 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 428.257606] nixbot[3632]: INFO: ::ffff:127.0.0.1:58454 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 429.311627] nixbot[3632]: INFO: ::ffff:127.0.0.1:58460 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 430.368857] nixbot[3632]: INFO: ::ffff:127.0.0.1:58466 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 431.422760] nixbot[3632]: INFO: ::ffff:127.0.0.1:34838 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 431.639447] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:28.510Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 431.642622] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:28.513Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 431.650797] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:28.513Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 431.656554] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:28.513Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 431.662804] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:28.517Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 432.479548] nixbot[3632]: INFO: ::ffff:127.0.0.1:34848 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 433.538361] nixbot[3632]: INFO: ::ffff:127.0.0.1:34856 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 434.437172] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:31.307Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"627d13e651d8aff80c3ecca4","created_at":"2026-08-27T02:06:58.442Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:58.520Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-627d13e651d8aff80c3ecca4: start","job_status":"start","queue_duration_s":92.786918,"scheduling_latency_s":92.78697} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 434.582051] nixbot[3632]: INFO: ::ffff:127.0.0.1:34860 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.04 seconds) gitlab # [ 434.770694] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:31.641Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"627d13e651d8aff80c3ecca4","created_at":"2026-08-27T02:06:58.442Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:58.520Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-627d13e651d8aff80c3ecca4: done: 0.334025 sec","job_status":"done","queue_duration_s":92.786918,"scheduling_latency_s":92.78697,"redis_calls":23,"redis_duration_s":0.009415,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.004745,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002836,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001834,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.182143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.20919502,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.334025,"completed_at":"2026-08-27T02:08:31.641Z","load_balancing_strategy":"primary","db_duration_s":0.109082,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 434.883492] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:31.645Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f6c3b7dad3bebd1a0292a7cf","created_at":"2026-08-27T02:06:58.555Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:58.568Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f6c3b7dad3bebd1a0292a7cf: start","job_status":"start","queue_duration_s":93.077279,"scheduling_latency_s":93.077329} gitlab # [ 434.969819] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:31.840Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f6c3b7dad3bebd1a0292a7cf","created_at":"2026-08-27T02:06:58.555Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:58.568Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f6c3b7dad3bebd1a0292a7cf: done: 0.195234 sec","job_status":"done","queue_duration_s":93.077279,"scheduling_latency_s":93.077329,"redis_calls":14,"redis_duration_s":0.006235,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001284,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002926,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002025,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.142792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1448222,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.195234,"completed_at":"2026-08-27T02:08:31.840Z","load_balancing_strategy":"primary","db_duration_s":0.039259,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 435.066890] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:31.843Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"2c0f538e36c3b5c1b3e82a94","created_at":"2026-08-27T02:06:59.652Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:59.663Z","job_size_bytes":5,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-2c0f538e36c3b5c1b3e82a94: start","job_status":"start","queue_duration_s":92.179739,"scheduling_latency_s":92.17977} gitlab # [ 435.211916] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:32.082Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"2c0f538e36c3b5c1b3e82a94","created_at":"2026-08-27T02:06:59.652Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:59.663Z","job_size_bytes":5,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-2c0f538e36c3b5c1b3e82a94: done: 0.23949 sec","job_status":"done","queue_duration_s":92.179739,"scheduling_latency_s":92.17977,"redis_calls":12,"redis_duration_s":0.004063000000000001,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001051,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002354,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000658,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.186818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042448,"duration_s":0.23949,"completed_at":"2026-08-27T02:08:32.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.045729,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 435.318148] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:32.084Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"0ec6d729a75ee2f11962220c","created_at":"2026-08-27T02:06:59.713Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:59.726Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-0ec6d729a75ee2f11962220c: start","job_status":"start","queue_duration_s":92.358132,"scheduling_latency_s":92.358151} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.07 seconds) gitlab # [ 435.608135] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:32.477Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"0ec6d729a75ee2f11962220c","created_at":"2026-08-27T02:06:59.713Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:59.726Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-0ec6d729a75ee2f11962220c: done: 0.392877 sec","job_status":"done","queue_duration_s":92.358132,"scheduling_latency_s":92.358151,"redis_calls":11,"redis_duration_s":0.0036349999999999998,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000842,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000674,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.342,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.342,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.28041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00079536,"duration_s":0.392877,"completed_at":"2026-08-27T02:08:32.477Z","load_balancing_strategy":"primary","db_duration_s":0.120956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.711762] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:32.482Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"a0d36283cbd7857513ff03d1","created_at":"2026-08-27T02:06:59.735Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:59.748Z","job_size_bytes":89,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-a0d36283cbd7857513ff03d1: start","job_status":"start","queue_duration_s":92.733828,"scheduling_latency_s":92.733858} gitlab # [ 435.734876] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:32.549Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"a0d36283cbd7857513ff03d1","created_at":"2026-08-27T02:06:59.735Z","trace_propagation_headers":{"sentry-trace":"99a939b2fe0b4a08813891e591dc2d1c-a6078b22cce44fcd","baggage":"sentry-trace_id=99a939b2fe0b4a08813891e591dc2d1c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGSEGN1Q8B0BWNHVMHX7M","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-08-27T02:06:59.748Z","job_size_bytes":89,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-a0d36283cbd7857513ff03d1: done: 0.067899 sec","job_status":"done","queue_duration_s":92.733828,"scheduling_latency_s":92.733858,"redis_calls":11,"redis_duration_s":0.003884,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001067,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000711,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052796,"duration_s":0.067899,"completed_at":"2026-08-27T02:08:32.549Z","load_balancing_strategy":"primary","db_duration_s":0.01272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.784293] nixbot[3632]: INFO: ::ffff:127.0.0.1:34872 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 435.786670] gitaly[906]: time="2026-08-27T02:08:32.637Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:42.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:32.620" grpc.time_ms=17.759 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.800344] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:32.552Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 9e7da772ac2fa6f1071cffc107848b72a7804eb7 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"6ccd0dce7d0f5c759c24fe5c","created_at":"2026-08-27T02:07:02.058Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-7870de12393a4d8d","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:07:02.068Z","job_size_bytes":185,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-6ccd0dce7d0f5c759c24fe5c: start","job_status":"start","queue_duration_s":90.484751,"scheduling_latency_s":90.484778} gitlab # [ 435.817446] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:32.578Z","message":"changes: \"0000000000000000000000000000000000000000 9e7da772ac2fa6f1071cffc107848b72a7804eb7 refs/heads/master\\n\""} gitlab # [ 435.819870] gitaly[906]: time="2026-08-27T02:08:32.662Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-27T02:08:42.035" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:32.645" grpc.time_ms=16.615 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.873211] gitaly[906]: time="2026-08-27T02:08:32.744Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-08-27T02:08:42.120" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:32.731" grpc.time_ms=13.315 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.951456] gitaly[906]: time="2026-08-27T02:08:32.823Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:32.822" grpc.time_ms=0.706 pid=906 span.kind=server system=grpc gitlab # [ 436.007516] gitaly[906]: time="2026-08-27T02:08:32.879Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:32.877" grpc.time_ms=1.119 pid=906 span.kind=server system=grpc gitlab # [ 436.036945] gitaly[906]: time="2026-08-27T02:08:32.891Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 diskcache=77345a5a-bbfe-4916-9c09-936d95ebe737 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-27T02:08:42.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:32.767" pid=906 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 436.058679] gitaly[906]: time="2026-08-27T02:08:32.892Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=135 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5087 command.oublock=8 command.real_time_ms=118 command.spawn_token_fork_ms=1 command.system_time_ms=68 command.user_time_ms=67 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-27T02:08:42.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:32.767" grpc.time_ms=124.248 pid=906 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 436.082910] gitaly[906]: time="2026-08-27T02:08:32.927Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-27T02:08:42.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:32.926" grpc.time_ms=0.31 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.100681] gitaly[906]: time="2026-08-27T02:08:32.959Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=858 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-08-27T02:08:42.014" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:32.935" grpc.time_ms=23.479 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.274730] gitaly[906]: time="2026-08-27T02:08:33.146Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-08-27T02:09:03.033" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:33.133" grpc.time_ms=12.13 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.336445] gitaly[906]: time="2026-08-27T02:08:33.208Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-27T02:08:43.006" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:33.207" grpc.time_ms=0.578 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.368806] gitaly[906]: time="2026-08-27T02:08:33.222Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-08-27T02:08:43.011" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-08-27T02:08:33.212" grpc.time_ms=10.489 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.399058] gitaly[906]: time="2026-08-27T02:08:33.263Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:43.048" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:33.248" grpc.time_ms=15.214 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.419992] gitaly[906]: time="2026-08-27T02:08:33.288Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:43.071" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:33.271" grpc.time_ms=15.997 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.463815] gitaly[906]: time="2026-08-27T02:08:33.335Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=2 command.system_time_ms=17 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-27T02:09:03.206" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:33.307" grpc.time_ms=28.178 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 436.720395] nixbot[3632]: INFO: ::ffff:127.0.0.1:34874 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.07 seconds) gitlab # [ 437.133773] gitaly[906]: time="2026-08-27T02:08:34.004Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-08-27T02:08:44.001" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:34.001" grpc.time_ms=2.001 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.180500] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:34.050Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 9e7da772ac2fa6f1071cffc107848b72a7804eb7 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"6ccd0dce7d0f5c759c24fe5c","created_at":"2026-08-27T02:07:02.058Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-7870de12393a4d8d","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:07:02.068Z","job_size_bytes":185,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-6ccd0dce7d0f5c759c24fe5c: done: 1.497663 sec","job_status":"done","queue_duration_s":90.484751,"scheduling_latency_s":90.484778,"gitaly_calls":13,"gitaly_duration_s":0.335594,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.071377,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.006354,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.009172,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.003051,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005978,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.028,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.018822,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.094,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.337,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.389,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.094,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.00319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00106402,"exclusive_lock_hold_duration_s":0.04409114,"duration_s":1.497663,"completed_at":"2026-08-27T02:08:34.050Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.166265,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 437.276487] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:34.052Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","9e7da772ac2fa6f1071cffc107848b72a7804eb7 029b33b836639f8d58dd1784db28a8cf810aca0d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a103816a0a1f968087e12eaf","created_at":"2026-08-27T02:07:07.741Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-0b12ed8417c14619","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:07:07.745Z","job_size_bytes":185,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a103816a0a1f968087e12eaf: start","job_status":"start","queue_duration_s":86.307164,"scheduling_latency_s":86.307193} gitlab # [ 437.296157] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:34.068Z","message":"changes: \"9e7da772ac2fa6f1071cffc107848b72a7804eb7 029b33b836639f8d58dd1784db28a8cf810aca0d refs/heads/master\\n\""} gitlab # [ 437.298806] gitaly[906]: time="2026-08-27T02:08:34.111Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:44.091" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:34.091" grpc.time_ms=19.23 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.314717] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:34.149Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"4bdadf91df3e7d568ff65f3b","created_at":"2026-08-27T02:08:34.145Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-c1dec1f8ecd346c8","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"c5c14370cd12e9d0314e5a68","job_size_bytes":46,"pid":3569,"job_status":"deduplicated","message":"ProjectCacheWorker JID-4bdadf91df3e7d568ff65f3b: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 437.331337] gitaly[906]: time="2026-08-27T02:08:34.139Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-27T02:08:44.122" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:34.122" grpc.time_ms=16.43 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.346404] gitaly[906]: time="2026-08-27T02:08:34.200Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-08-27T02:08:44.180" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:34.181" grpc.time_ms=18.902 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.361435] gitaly[906]: time="2026-08-27T02:08:34.232Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-08-27T02:09:04.111" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:34.212" grpc.time_ms=20.616 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.396075] gitaly[906]: time="2026-08-27T02:08:34.267Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-08-27T02:08:44.055" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-08-27T02:08:34.255" grpc.time_ms=11.624 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.437785] gitaly[906]: time="2026-08-27T02:08:34.309Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:44.089" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:34.290" grpc.time_ms=18.767 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.673046] gitaly[906]: time="2026-08-27T02:08:34.544Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-08-27T02:08:44.010" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:34.541" grpc.time_ms=3.095 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 437.724490] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:34.594Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","9e7da772ac2fa6f1071cffc107848b72a7804eb7 029b33b836639f8d58dd1784db28a8cf810aca0d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a103816a0a1f968087e12eaf","created_at":"2026-08-27T02:07:07.741Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-0b12ed8417c14619","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:07:07.745Z","job_size_bytes":185,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a103816a0a1f968087e12eaf: done: 0.541999 sec","job_status":"done","queue_duration_s":86.307164,"scheduling_latency_s":86.307193,"gitaly_calls":7,"gitaly_duration_s":0.143905,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.043585,"redis_read_bytes":500,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.005151,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.002382,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006044,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.012886,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.017122,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.058,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.058,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.344536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00098568,"exclusive_lock_hold_duration_s":0.03387178,"duration_s":0.541999,"completed_at":"2026-08-27T02:08:34.594Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05474,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 437.835009] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:34.598Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ad8b3b58bbd78f202b9ad443","created_at":"2026-08-27T02:07:08.071Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-a800d93f85a947ed","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-08-27T02:07:08.083Z","job_size_bytes":65,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ad8b3b58bbd78f202b9ad443: start","job_status":"start","queue_duration_s":86.51479,"scheduling_latency_s":86.514836} gitlab # [ 437.858603] nixbot[3632]: INFO: ::ffff:127.0.0.1:34888 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 437.953543] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:34.824Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ad8b3b58bbd78f202b9ad443","created_at":"2026-08-27T02:07:08.071Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-a800d93f85a947ed","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-08-27T02:07:08.083Z","job_size_bytes":65,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ad8b3b58bbd78f202b9ad443: done: 0.225665 sec","job_status":"done","queue_duration_s":86.51479,"scheduling_latency_s":86.514836,"redis_calls":16,"redis_duration_s":0.009923000000000001,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.005262,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00336,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001301,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.160835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00104478,"duration_s":0.225665,"completed_at":"2026-08-27T02:08:34.823Z","load_balancing_strategy":"primary","db_duration_s":0.051427,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 438.052625] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:34.827Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d6c9b39b2701e12651201be1","created_at":"2026-08-27T02:07:08.086Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-a800d93f85a947ed","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-08-27T02:07:08.095Z","job_size_bytes":65,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d6c9b39b2701e12651201be1: start","job_status":"start","queue_duration_s":86.732533,"scheduling_latency_s":86.732576} gitlab # [ 438.310578] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:35.180Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d6c9b39b2701e12651201be1","created_at":"2026-08-27T02:07:08.086Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-a800d93f85a947ed","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-08-27T02:07:08.095Z","job_size_bytes":65,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d6c9b39b2701e12651201be1: done: 0.35298 sec","job_status":"done","queue_duration_s":86.732533,"scheduling_latency_s":86.732576,"redis_calls":21,"redis_duration_s":0.009111000000000001,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.004645,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002259,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.279729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005617,"duration_s":0.35298,"completed_at":"2026-08-27T02:08:35.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.412317] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:35.182Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9e80fc3a1a427651d136e524","created_at":"2026-08-27T02:07:08.262Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-a800d93f85a947ed","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-27T02:07:08.270Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9e80fc3a1a427651d136e524: start","job_status":"start","queue_duration_s":86.912867,"scheduling_latency_s":86.912895} gitlab # [ 438.702095] gitaly[906]: time="2026-08-27T02:08:35.572Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:35.571" grpc.time_ms=0.937 pid=906 span.kind=server system=grpc gitlab # [ 438.753092] gitaly[906]: time="2026-08-27T02:08:35.624Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:35.623" grpc.time_ms=0.862 pid=906 span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 438.819471] gitaly[906]: time="2026-08-27T02:08:35.690Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:35.689" grpc.time_ms=0.819 pid=906 span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 438.848330] gitaly[906]: time="2026-08-27T02:08:35.707Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 diskcache=00b75d65-1f1a-4b09-bdec-6da2eead5397 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:05.411" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:35.511" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.872221] gitaly[906]: time="2026-08-27T02:08:35.708Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=212 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7606 command.oublock=0 command.real_time_ms=188 command.spawn_token_fork_ms=1 command.system_time_ms=123 command.user_time_ms=89 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:05.411" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:35.511" grpc.time_ms=195.791 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.896895] nixbot[3632]: INFO: ::ffff:127.0.0.1:34894 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 439.098448] gitaly[906]: time="2026-08-27T02:08:35.969Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-27T02:09:05.867" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:35.967" grpc.time_ms=1.673 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.196297] gitaly[906]: time="2026-08-27T02:08:36.067Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH4RVS92M7Z11ZTWRJMG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:06.065" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:36.065" grpc.time_ms=1.477 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.251220] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:36.121Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9e80fc3a1a427651d136e524","created_at":"2026-08-27T02:07:08.262Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-a800d93f85a947ed","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-27T02:07:08.270Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9e80fc3a1a427651d136e524: done: 0.938714 sec","job_status":"done","queue_duration_s":86.912867,"scheduling_latency_s":86.912895,"gitaly_calls":3,"gitaly_duration_s":0.217038,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.058281,"redis_read_bytes":457,"redis_write_bytes":28001,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.003203,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002948,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.020182,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.006676,"redis_queues_read_bytes":36,"redis_queues_write_bytes":14495,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.015434,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001482,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000972,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004181,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.108,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.056,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.063,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.058,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.576823,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.81790578,"duration_s":0.938714,"completed_at":"2026-08-27T02:08:36.121Z","load_balancing_strategy":"primary","db_duration_s":0.128686,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 439.355064] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:36.125Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c5c14370cd12e9d0314e5a68","created_at":"2026-08-27T02:08:32.666Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:32.691Z","job_size_bytes":46,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c5c14370cd12e9d0314e5a68: start","job_status":"start","queue_duration_s":3.433489,"scheduling_latency_s":3.433537} gitlab # [ 439.376428] gitaly[906]: time="2026-08-27T02:08:36.238Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-08-27T08:08:36.226" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:36.227" grpc.time_ms=10.118 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.400291] gitaly[906]: time="2026-08-27T02:08:36.271Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=594 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-08-27T02:09:06.253" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:36.253" grpc.time_ms=17.773 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.454676] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:36.325Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c5c14370cd12e9d0314e5a68","created_at":"2026-08-27T02:08:32.666Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:32.691Z","job_size_bytes":46,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c5c14370cd12e9d0314e5a68: done: 0.200531 sec","job_status":"done","queue_duration_s":3.433489,"scheduling_latency_s":3.433537,"gitaly_calls":2,"gitaly_duration_s":0.036855,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011956,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002468,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000996,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00253,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004704,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.133274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047518,"duration_s":0.200531,"completed_at":"2026-08-27T02:08:36.325Z","load_balancing_strategy":"primary","db_duration_s":0.01985,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 439.533122] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:36.327Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a13d21ba6485e5ec5a9cfd53","created_at":"2026-08-27T02:08:32.717Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:32.725Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a13d21ba6485e5ec5a9cfd53: start","job_status":"start","queue_duration_s":3.602837,"scheduling_latency_s":3.602861} gitlab # [ 439.559115] gitaly[906]: time="2026-08-27T02:08:36.396Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 diskcache=efbe5a3b-374a-4d83-a229-9bf2b319c3b6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-27T08:08:36.360" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:36.361" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.573701] gitaly[906]: time="2026-08-27T02:08:36.397Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=41 command.spawn_token_fork_ms=3 command.system_time_ms=19 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-27T08:08:36.360" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:36.361" grpc.time_ms=35.61 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.593854] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:36.459Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a13d21ba6485e5ec5a9cfd53","created_at":"2026-08-27T02:08:32.717Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:32.725Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a13d21ba6485e5ec5a9cfd53: done: 0.131601 sec","job_status":"done","queue_duration_s":3.602837,"scheduling_latency_s":3.602861,"gitaly_calls":1,"gitaly_duration_s":0.039583,"redis_calls":15,"redis_duration_s":0.004976,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001858,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000992,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001553,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.017,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.032,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09723458,"duration_s":0.131601,"completed_at":"2026-08-27T02:08:36.459Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.646869] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:36.461Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"69b8b7271a503578c51db76d","created_at":"2026-08-27T02:08:32.917Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:32.921Z","job_size_bytes":23,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-69b8b7271a503578c51db76d: start","job_status":"start","queue_duration_s":3.540207,"scheduling_latency_s":3.540232} gitlab # [ 439.664496] gitaly[906]: time="2026-08-27T02:08:36.531Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-08-27T02:09:06.514" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:36.515" grpc.time_ms=16.111 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.684518] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:36.554Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"69b8b7271a503578c51db76d","created_at":"2026-08-27T02:08:32.917Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:32.921Z","job_size_bytes":23,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-69b8b7271a503578c51db76d: done: 0.093198 sec","job_status":"done","queue_duration_s":3.540207,"scheduling_latency_s":3.540232,"gitaly_calls":1,"gitaly_duration_s":0.021814,"redis_calls":16,"redis_duration_s":0.0060149999999999995,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000534,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001709,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002676,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.093198,"completed_at":"2026-08-27T02:08:36.554Z","load_balancing_strategy":"primary","db_duration_s":0.005666,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 439.731759] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:36.558Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"eb8f450bc4ef9a8bc11b920e","created_at":"2026-08-27T02:08:33.761Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2a4d7c88607ea480ec27a6210e1f4bd5a76c0decf8b60096f991d1e32b36480","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:33.771Z","job_size_bytes":467,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-eb8f450bc4ef9a8bc11b920e: start","job_status":"start","queue_duration_s":2.787062,"scheduling_latency_s":2.787103} gitlab # [ 439.751411] gitaly[906]: time="2026-08-27T02:08:36.610Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:06.608" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:36.608" grpc.time_ms=1.23 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.764615] gitaly[906]: time="2026-08-27T02:08:36.630Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-27T02:08:46.084" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:36.614" grpc.time_ms=15.312 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.07 seconds) gitlab # [ 439.918031] nixbot[3632]: INFO: ::ffff:127.0.0.1:34908 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 439.959979] gitaly[906]: time="2026-08-27T02:08:36.831Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-27T02:09:06.829" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:36.829" grpc.time_ms=1.403 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.975388] gitaly[906]: time="2026-08-27T02:08:36.846Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=54 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-08-27T02:09:06.840" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:36.840" grpc.time_ms=5.466 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 440.176704] gitaly[906]: time="2026-08-27T02:08:37.048Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:47.029" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:37.029" grpc.time_ms=17.982 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 440.975699] nixbot[3632]: INFO: ::ffff:127.0.0.1:44854 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.06 seconds) gitlab # [ 441.077703] gitaly[906]: time="2026-08-27T02:08:37.948Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=2 command.system_time_ms=18 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-27T02:09:07.018" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:37.919" grpc.time_ms=28.811 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.122402] gitaly[906]: time="2026-08-27T02:08:37.980Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FGY7BS202X1T11VQVMM02 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-08-27T02:09:07.052" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:37.952" grpc.time_ms=27.429 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.159026] sidekiq-cluster[3569]: {"severity":"WARN","time":"2026-08-27T02:08:38.029Z","class":"CreatePipelineWorker","correlation_id":"01M10FGY7BS202X1T11VQVMM02","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"9e7da772ac2fa6f1071cffc107848b72a7804eb7","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"9e7da772ac2fa6f1071cffc107848b72a7804eb7","ref":"refs/heads/master"},"retry":0} gitlab # [ 441.173215] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:38.041Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"eb8f450bc4ef9a8bc11b920e","created_at":"2026-08-27T02:08:33.761Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2a4d7c88607ea480ec27a6210e1f4bd5a76c0decf8b60096f991d1e32b36480","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:33.771Z","job_size_bytes":467,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-eb8f450bc4ef9a8bc11b920e: done: 1.48318 sec","job_status":"done","queue_duration_s":2.787062,"scheduling_latency_s":2.787103,"gitaly_calls":7,"gitaly_duration_s":0.126473,"redis_calls":51,"redis_duration_s":0.016379,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.001507,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.00719,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004196,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001568,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.269196,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054402,"duration_s":1.48318,"completed_at":"2026-08-27T02:08:38.041Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.094382,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 441.261160] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:38.044Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"de79f3ad-c313-4c05-a466-7c7d31ba02d4\"}"],"class":"WebHookWorker","jid":"3ce8034ae391e298891ba4f9","created_at":"2026-08-27T02:08:33.956Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d9e293d0dff5b86b3c232840f175bd77bb5b6232f7b61ae534754e8df7d79efb","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:33.964Z","job_size_bytes":1751,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3ce8034ae391e298891ba4f9: start","job_status":"start","queue_duration_s":4.079635,"scheduling_latency_s":4.079669} gitlab # [ 441.350569] nixbot[3632]: INFO: ::ffff:127.0.0.1:44858 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 441.395165] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:38.265Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"de79f3ad-c313-4c05-a466-7c7d31ba02d4\"}"],"class":"WebHookWorker","jid":"3ce8034ae391e298891ba4f9","created_at":"2026-08-27T02:08:33.956Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-ec6dc0952c114504","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d9e293d0dff5b86b3c232840f175bd77bb5b6232f7b61ae534754e8df7d79efb","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:33.964Z","job_size_bytes":1751,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3ce8034ae391e298891ba4f9: done: 0.221429 sec","job_status":"done","queue_duration_s":4.079635,"scheduling_latency_s":4.079669,"redis_calls":19,"redis_duration_s":0.007932999999999999,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002202,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000751,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003332,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001648,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.07270255999998199,"cpu_s":0.127361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0010732,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.221429,"completed_at":"2026-08-27T02:08:38.265Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.504897] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:38.269Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"42f93c94aab306893a182fdf","created_at":"2026-08-27T02:08:34.165Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-c1dec1f8ecd346c8","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"a13d21ba6485e5ec5a9cfd53","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:34.171Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-42f93c94aab306893a182fdf: start","job_status":"start","queue_duration_s":4.097492,"scheduling_latency_s":4.097537} gitlab # [ 441.527994] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:38.368Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"42f93c94aab306893a182fdf","created_at":"2026-08-27T02:08:34.165Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-c1dec1f8ecd346c8","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"a13d21ba6485e5ec5a9cfd53","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:34.171Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-42f93c94aab306893a182fdf: done: 0.099802 sec","job_status":"done","queue_duration_s":4.097492,"scheduling_latency_s":4.097537,"gitaly_calls":1,"gitaly_duration_s":0.024651,"redis_calls":12,"redis_duration_s":0.008308,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000764,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05761432,"duration_s":0.099802,"completed_at":"2026-08-27T02:08:38.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.578702] gitaly[906]: time="2026-08-27T02:08:38.327Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 diskcache=aeae0d5f-b6b3-4235-93cf-e7df3437c0bd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-27T08:08:38.307" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:38.308" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.590617] gitaly[906]: time="2026-08-27T02:08:38.327Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-27T08:08:38.307" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:38.308" grpc.time_ms=19.538 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.605174] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:38.374Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"328c8d9f7a72c6bfa73110ba","created_at":"2026-08-27T02:08:34.444Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-c1dec1f8ecd346c8","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ee71362f86e9f925e0ea1863eae9ceb3c885f1351dd282c2d3e3f0a4abe97d1b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:34.452Z","job_size_bytes":467,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-328c8d9f7a72c6bfa73110ba: start","job_status":"start","queue_duration_s":3.922182,"scheduling_latency_s":3.922219} gitlab # [ 441.624672] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 157 gitlab # [ 441.626942] gitaly[906]: time="2026-08-27T02:08:38.421Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FKXBTEHX387EERKEV7JF1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:38.406" grpc.time_ms=15.127 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 441.641361] gitaly[906]: time="2026-08-27T02:08:38.425Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:08.422" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:38.422" grpc.time_ms=2.73 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.654796] gitaly[906]: time="2026-08-27T02:08:38.445Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-27T02:08:48.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:38.429" grpc.time_ms=15.633 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.669166] gitaly[906]: time="2026-08-27T02:08:38.505Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-27T02:09:08.503" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:38.503" grpc.time_ms=1.321 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.682924] gitlab-workhorse[824]: time="2026-08-27T02:08:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 441.684969] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 102 gitlab # [ 441.686975] gitaly[906]: time="2026-08-27T02:08:38.521Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=203 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-08-27T02:09:08.511" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:38.512" grpc.time_ms=8.505 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.704490] gitaly[906]: time="2026-08-27T02:08:38.525Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FKXGT92HJ7QM25Y3YNEK9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:38.511" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 441.715371] gitaly[906]: time="2026-08-27T02:08:38.525Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKXGT92HJ7QM25Y3YNEK9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:38.511" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 441.730188] nixbot[3632]: {"ts": "2026-08-27T02:08:38.586Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 441.732851] gitaly[906]: time="2026-08-27T02:08:38.526Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKXGT92HJ7QM25Y3YNEK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:38.511" grpc.time_ms=15.183 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 441.748431] gitaly[906]: time="2026-08-27T02:08:38.579Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:48.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:38.565" grpc.time_ms=14.235 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.802505] gitaly[906]: time="2026-08-27T02:08:38.674Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:08.672" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:38.672" grpc.time_ms=0.883 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 441.936549] gitaly[906]: time="2026-08-27T02:08:38.808Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:48.229" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:38.789" grpc.time_ms=17.878 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 442.053741] nixbot[3632]: INFO: ::ffff:127.0.0.1:44862 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.08 seconds) gitlab # [ 442.064569] nixbot[3632]: {"ts": "2026-08-27T02:08:38.933Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 \"HTTP/1.1 201 Created\""} gitlab # [ 442.069834] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 322 gitlab # [ 442.191605] gitaly[906]: time="2026-08-27T02:08:39.061Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKY1046QQGXHBPMKRTBHR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:09.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:39.060" grpc.time_ms=1.281 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.291805] gitaly[906]: time="2026-08-27T02:08:39.163Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FKY1046QQGXHBPMKRTBHR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:49.143" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:39.144" grpc.time_ms=18.315 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.450602] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 376 gitlab # [ 442.456379] nixbot[3632]: {"ts": "2026-08-27T02:08:39.325Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 \"HTTP/1.1 201 Created\""} gitlab # [ 442.493094] gitaly[906]: time="2026-08-27T02:08:39.364Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=7 command.system_time_ms=16 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-27T02:09:09.335" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:39.336" grpc.time_ms=28.247 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.522081] gitaly[906]: time="2026-08-27T02:08:39.391Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FH402T6CB0A36PR2ERK51 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-08-27T02:09:09.368" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:39.369" grpc.time_ms=22.389 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.544590] sidekiq-cluster[3569]: {"severity":"WARN","time":"2026-08-27T02:08:39.415Z","class":"CreatePipelineWorker","correlation_id":"01M10FH402T6CB0A36PR2ERK51","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"9e7da772ac2fa6f1071cffc107848b72a7804eb7","after":"029b33b836639f8d58dd1784db28a8cf810aca0d","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"029b33b836639f8d58dd1784db28a8cf810aca0d","ref":"refs/heads/master"},"retry":0} gitlab # [ 442.554114] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.423Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"328c8d9f7a72c6bfa73110ba","created_at":"2026-08-27T02:08:34.444Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-c1dec1f8ecd346c8","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ee71362f86e9f925e0ea1863eae9ceb3c885f1351dd282c2d3e3f0a4abe97d1b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:34.452Z","job_size_bytes":467,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-328c8d9f7a72c6bfa73110ba: done: 1.048901 sec","job_status":"done","queue_duration_s":3.922182,"scheduling_latency_s":3.922219,"gitaly_calls":7,"gitaly_duration_s":0.118969,"redis_calls":33,"redis_duration_s":0.013807,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001608,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002632,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004326,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005241,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.88527,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.048901,"completed_at":"2026-08-27T02:08:39.423Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043542,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 442.616658] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.425Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"f6941e8e-7ff1-4c92-8ad1-05791039510e\"}"],"class":"WebHookWorker","jid":"217dfa13f4ab76c8b701e53f","created_at":"2026-08-27T02:08:34.497Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-c1dec1f8ecd346c8","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1fa164c8a9c13b8ec9114be1b7b04dbb7f72dedf7b249d171c111dcda71dbf90","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:34.504Z","job_size_bytes":1749,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-217dfa13f4ab76c8b701e53f: start","job_status":"start","queue_duration_s":4.921377,"scheduling_latency_s":4.92141} gitlab # [ 442.636814] nixbot[3632]: INFO: ::ffff:127.0.0.1:44864 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 442.638147] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.497Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"f6941e8e-7ff1-4c92-8ad1-05791039510e\"}"],"class":"WebHookWorker","jid":"217dfa13f4ab76c8b701e53f","created_at":"2026-08-27T02:08:34.497Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-c1dec1f8ecd346c8","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1fa164c8a9c13b8ec9114be1b7b04dbb7f72dedf7b249d171c111dcda71dbf90","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:34.504Z","job_size_bytes":1749,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-217dfa13f4ab76c8b701e53f: done: 0.071509 sec","job_status":"done","queue_duration_s":4.921377,"scheduling_latency_s":4.92141,"redis_calls":14,"redis_duration_s":0.009367,"redis_read_bytes":26,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.001975,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005761,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001631,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.013175139999987096,"cpu_s":0.038938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.071509,"completed_at":"2026-08-27T02:08:39.497Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004213,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.692465] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.508Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"15d799397051a01b762dfa35","created_at":"2026-08-27T02:08:35.471Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.484Z","job_size_bytes":23,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-15d799397051a01b762dfa35: start","job_status":"start","queue_duration_s":4.023956,"scheduling_latency_s":4.024009} gitlab # [ 442.710286] gitaly[906]: time="2026-08-27T02:08:39.555Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKYD81PYS2ND52R1R2S99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:09.548" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:39.549" grpc.time_ms=1.115 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.766601] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.637Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"15d799397051a01b762dfa35","created_at":"2026-08-27T02:08:35.471Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.484Z","job_size_bytes":23,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-15d799397051a01b762dfa35: done: 0.129309 sec","job_status":"done","queue_duration_s":4.023956,"scheduling_latency_s":4.024009,"redis_calls":10,"redis_duration_s":0.005504,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000763,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003517,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00075512,"duration_s":0.129309,"completed_at":"2026-08-27T02:08:39.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019736,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 442.834504] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.641Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"dbf55b8c7223ad0a078ad045","created_at":"2026-08-27T02:08:35.720Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.735Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-dbf55b8c7223ad0a078ad045: start","job_status":"start","queue_duration_s":3.905456,"scheduling_latency_s":3.905895} gitlab # [ 442.856165] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.687Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"dbf55b8c7223ad0a078ad045","created_at":"2026-08-27T02:08:35.720Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.735Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-dbf55b8c7223ad0a078ad045: done: 0.045804 sec","job_status":"done","queue_duration_s":3.905456,"scheduling_latency_s":3.905895,"redis_calls":10,"redis_duration_s":0.00432,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002548,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003354,"duration_s":0.045804,"completed_at":"2026-08-27T02:08:39.687Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005013,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.904201] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 128 gitlab # [ 442.906452] gitaly[906]: time="2026-08-27T02:08:39.653Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FKYK5GKX4R1K14EQQBHVJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:39.635" grpc.time_ms=18.041 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 442.920115] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.689Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8c86f67892e59b7efe168432","created_at":"2026-08-27T02:08:35.761Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.782Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8c86f67892e59b7efe168432: start","job_status":"start","queue_duration_s":3.906945,"scheduling_latency_s":3.906972} gitlab # [ 442.936700] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.745Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8c86f67892e59b7efe168432","created_at":"2026-08-27T02:08:35.761Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.782Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8c86f67892e59b7efe168432: done: 0.055164 sec","job_status":"done","queue_duration_s":3.906945,"scheduling_latency_s":3.906972,"redis_calls":10,"redis_duration_s":0.003522,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001169,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035348,"duration_s":0.055164,"completed_at":"2026-08-27T02:08:39.745Z","load_balancing_strategy":"primary","db_duration_s":0.006991,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.982417] gitaly[906]: time="2026-08-27T02:08:39.736Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FKYD81PYS2ND52R1R2S99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:49.011" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:39.721" grpc.time_ms=13.935 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.997111] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.746Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"746ce19ca0a1b1018d8f1170","created_at":"2026-08-27T02:08:35.815Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.830Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-746ce19ca0a1b1018d8f1170: start","job_status":"start","queue_duration_s":3.916136,"scheduling_latency_s":3.916158} gitlab # [ 443.013128] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.795Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"746ce19ca0a1b1018d8f1170","created_at":"2026-08-27T02:08:35.815Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.830Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-746ce19ca0a1b1018d8f1170: done: 0.048896 sec","job_status":"done","queue_duration_s":3.916136,"scheduling_latency_s":3.916158,"redis_calls":13,"redis_duration_s":0.005618000000000001,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002814,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000872,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029588,"duration_s":0.048896,"completed_at":"2026-08-27T02:08:39.795Z","load_balancing_strategy":"primary","db_duration_s":0.005296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 443.063868] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.797Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f4c9ebf5afeaa017f2db213c","created_at":"2026-08-27T02:08:35.832Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.843Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f4c9ebf5afeaa017f2db213c: start","job_status":"start","queue_duration_s":3.953838,"scheduling_latency_s":3.953861} gitlab # [ 443.080717] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.834Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f4c9ebf5afeaa017f2db213c","created_at":"2026-08-27T02:08:35.832Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.843Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f4c9ebf5afeaa017f2db213c: done: 0.037441 sec","job_status":"done","queue_duration_s":3.953838,"scheduling_latency_s":3.953861,"redis_calls":10,"redis_duration_s":0.002081,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000336,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003428,"duration_s":0.037441,"completed_at":"2026-08-27T02:08:39.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.131401] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.836Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c9b2ddd56bab2af5075f9c7e","created_at":"2026-08-27T02:08:35.844Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.856Z","job_size_bytes":19,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c9b2ddd56bab2af5075f9c7e: start","job_status":"start","queue_duration_s":3.979714,"scheduling_latency_s":3.979738} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.12 seconds) gitlab # [ 443.148270] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.882Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c9b2ddd56bab2af5075f9c7e","created_at":"2026-08-27T02:08:35.844Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.856Z","job_size_bytes":19,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c9b2ddd56bab2af5075f9c7e: done: 0.045887 sec","job_status":"done","queue_duration_s":3.979714,"scheduling_latency_s":3.979738,"redis_calls":11,"redis_duration_s":0.0047610000000000005,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000667,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003393,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000701,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00062134,"duration_s":0.045887,"completed_at":"2026-08-27T02:08:39.882Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010189,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.193780] nixbot[3632]: {"ts": "2026-08-27T02:08:39.986Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 \"HTTP/1.1 201 Created\""} gitlab # [ 443.197005] nixbot[3632]: INFO: ::ffff:127.0.0.1:44878 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 443.199502] gitaly[906]: time="2026-08-27T02:08:39.950Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FKYQM9ETSWPWX4WP2PBJC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:39.928" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 443.211617] gitlab-workhorse[824]: time="2026-08-27T02:08:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 443.213660] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 283 gitlab # [ 443.215531] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e7da772ac2fa6f1071cffc107848b72a7804eb7 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 640 gitlab # [ 443.218264] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.884Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3abf0532bc9169f488397cec","created_at":"2026-08-27T02:08:35.864Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.874Z","job_size_bytes":66,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3abf0532bc9169f488397cec: start","job_status":"start","queue_duration_s":4.01013,"scheduling_latency_s":4.010157} gitlab # [ 443.237521] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.946Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3abf0532bc9169f488397cec","created_at":"2026-08-27T02:08:35.864Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.874Z","job_size_bytes":66,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3abf0532bc9169f488397cec: done: 0.062213 sec","job_status":"done","queue_duration_s":4.01013,"scheduling_latency_s":4.010157,"redis_calls":10,"redis_duration_s":0.0056170000000000005,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001717,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003261,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045334,"duration_s":0.062213,"completed_at":"2026-08-27T02:08:39.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.286337] gitaly[906]: time="2026-08-27T02:08:39.950Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKYQM9ETSWPWX4WP2PBJC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:39.928" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 443.301218] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 93 gitlab # [ 443.303440] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:39.949Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"46e2fcc77bb9d42c3a8f1086","created_at":"2026-08-27T02:08:35.876Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.885Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-46e2fcc77bb9d42c3a8f1086: start","job_status":"start","queue_duration_s":4.064371,"scheduling_latency_s":4.064399} gitlab # [ 443.321163] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.059Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"46e2fcc77bb9d42c3a8f1086","created_at":"2026-08-27T02:08:35.876Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.885Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-46e2fcc77bb9d42c3a8f1086: done: 0.109386 sec","job_status":"done","queue_duration_s":4.064371,"scheduling_latency_s":4.064399,"redis_calls":10,"redis_duration_s":0.009431,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001762,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":10,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00153678,"duration_s":0.109386,"completed_at":"2026-08-27T02:08:40.058Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.368664] gitaly[906]: time="2026-08-27T02:08:39.951Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKYQM9ETSWPWX4WP2PBJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:39.928" grpc.time_ms=23.093 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 443.384211] gitlab-workhorse[824]: time="2026-08-27T02:08:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 443.386131] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 86 gitlab # [ 443.388149] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.062Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"922eedd7d2e5148209d421e6","created_at":"2026-08-27T02:08:35.900Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.908Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-922eedd7d2e5148209d421e6: start","job_status":"start","queue_duration_s":4.153802,"scheduling_latency_s":4.153842} gitlab # [ 443.405601] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.132Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"922eedd7d2e5148209d421e6","created_at":"2026-08-27T02:08:35.900Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.908Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-922eedd7d2e5148209d421e6: done: 0.070008 sec","job_status":"done","queue_duration_s":4.153802,"scheduling_latency_s":4.153842,"redis_calls":10,"redis_duration_s":0.003486,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000673,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00174,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004464,"duration_s":0.070008,"completed_at":"2026-08-27T02:08:40.132Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.454193] gitaly[906]: time="2026-08-27T02:08:40.094Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FKZ22GT8ZEKYC3HMKCR9D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:40.083" grpc.time_ms=11.37 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 443.468439] gitaly[906]: time="2026-08-27T02:08:40.194Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZ5CY4KG4X95AW35QFNM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:40.180" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 443.479368] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.134Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c5f20a0b76033673fc9046f4","created_at":"2026-08-27T02:08:35.909Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.917Z","job_size_bytes":6,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c5f20a0b76033673fc9046f4: start","job_status":"start","queue_duration_s":4.216631,"scheduling_latency_s":4.216656} gitlab # [ 443.497983] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.174Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c5f20a0b76033673fc9046f4","created_at":"2026-08-27T02:08:35.909Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.917Z","job_size_bytes":6,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c5f20a0b76033673fc9046f4: done: 0.040645 sec","job_status":"done","queue_duration_s":4.216631,"scheduling_latency_s":4.216656,"redis_calls":10,"redis_duration_s":0.007604,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002737,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004183,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000684,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0008602,"duration_s":0.040645,"completed_at":"2026-08-27T02:08:40.174Z","load_balancing_strategy":"primary","db_duration_s":0.003018,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.544048] gitaly[906]: time="2026-08-27T02:08:40.194Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZ5CY4KG4X95AW35QFNM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:40.180" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 443.558413] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 77 gitlab # [ 443.561018] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.177Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a0455c4fd87a522019cb853f","created_at":"2026-08-27T02:08:35.918Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.926Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a0455c4fd87a522019cb853f: start","job_status":"start","queue_duration_s":4.250809,"scheduling_latency_s":4.250838} gitlab # [ 443.577535] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.245Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a0455c4fd87a522019cb853f","created_at":"2026-08-27T02:08:35.918Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.926Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a0455c4fd87a522019cb853f: done: 0.068136 sec","job_status":"done","queue_duration_s":4.250809,"scheduling_latency_s":4.250838,"redis_calls":10,"redis_duration_s":0.002431,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000566,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001359,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046852,"duration_s":0.068136,"completed_at":"2026-08-27T02:08:40.245Z","load_balancing_strategy":"primary","db_duration_s":0.020426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.624123] gitaly[906]: time="2026-08-27T02:08:40.194Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZ5CY4KG4X95AW35QFNM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:40.180" grpc.time_ms=14.353 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 443.639117] gitlab-workhorse[824]: time="2026-08-27T02:08:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 443.641112] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 124 gitlab # [ 443.643038] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.247Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"bd8ccf2ac0f9f930ab7cd02b","created_at":"2026-08-27T02:08:35.937Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.946Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-bd8ccf2ac0f9f930ab7cd02b: start","job_status":"start","queue_duration_s":4.301438,"scheduling_latency_s":4.301466} gitlab # [ 443.659762] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.375Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"bd8ccf2ac0f9f930ab7cd02b","created_at":"2026-08-27T02:08:35.937Z","trace_propagation_headers":{"sentry-trace":"63785fc81275494f87a7e4a57eedecb0-57ad4d04c2c74833","baggage":"sentry-trace_id=63785fc81275494f87a7e4a57eedecb0,sentry-release=Unknown"},"correlation_id":"01M10FH4RVS92M7Z11ZTWRJMG2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:35.946Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-bd8ccf2ac0f9f930ab7cd02b: done: 0.127652 sec","job_status":"done","queue_duration_s":4.301438,"scheduling_latency_s":4.301466,"redis_calls":14,"redis_duration_s":0.018823,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000572,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.013023,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00136484,"duration_s":0.127652,"completed_at":"2026-08-27T02:08:40.375Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.706484] gitaly[906]: time="2026-08-27T02:08:40.331Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FKZ9Z8TB7S6S2H2ST83P5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:40.319" grpc.time_ms=12.039 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 443.719905] gitaly[906]: time="2026-08-27T02:08:40.460Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZCG69CJB1JSC6M8FR08 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:40.439" pid=906 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 443.730551] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.377Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ad43f8c9058859e9aab2c193","created_at":"2026-08-27T02:08:36.291Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-43c4afe84e544a13","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:36.302Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ad43f8c9058859e9aab2c193: start","job_status":"start","queue_duration_s":4.074943,"scheduling_latency_s":4.074975} gitlab # [ 443.747052] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.452Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ad43f8c9058859e9aab2c193","created_at":"2026-08-27T02:08:36.291Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-43c4afe84e544a13","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:36.302Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ad43f8c9058859e9aab2c193: done: 0.074733 sec","job_status":"done","queue_duration_s":4.074943,"scheduling_latency_s":4.074975,"redis_calls":19,"redis_duration_s":0.010259,"redis_read_bytes":21,"redis_write_bytes":4288,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000996,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.001197,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2259,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003901,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004165,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01075264,"duration_s":0.074733,"completed_at":"2026-08-27T02:08:40.452Z","load_balancing_strategy":"primary","db_duration_s":0.010049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.793997] gitaly[906]: time="2026-08-27T02:08:40.460Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZCG69CJB1JSC6M8FR08 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:40.439" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 443.808407] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.454Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3ce8034ae391e298891ba4f9"],"class":"WebHooks::LogExecutionWorker","jid":"25ff67b29855744ca5865697","created_at":"2026-08-27T02:08:38.228Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-27017b34cecf4930","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e3fa5d66adf5eb8727d6411690e9def39fb5b844c17bf42c2fbe9d52e394a54a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:38.244Z","job_size_bytes":2593,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-25ff67b29855744ca5865697: start","job_status":"start","queue_duration_s":2.210275,"scheduling_latency_s":2.210307} gitlab # [ 443.825630] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.509Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3ce8034ae391e298891ba4f9"],"class":"WebHooks::LogExecutionWorker","jid":"25ff67b29855744ca5865697","created_at":"2026-08-27T02:08:38.228Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-27017b34cecf4930","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e3fa5d66adf5eb8727d6411690e9def39fb5b844c17bf42c2fbe9d52e394a54a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:38.244Z","job_size_bytes":2593,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-25ff67b29855744ca5865697: done: 0.05474 sec","job_status":"done","queue_duration_s":2.210275,"scheduling_latency_s":2.210307,"redis_calls":10,"redis_duration_s":0.002825,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000718,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002542,"duration_s":0.05474,"completed_at":"2026-08-27T02:08:40.509Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 443.873475] gitaly[906]: time="2026-08-27T02:08:40.460Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZCG69CJB1JSC6M8FR08 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:40.439" grpc.time_ms=21.342 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 443.889155] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.510Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fa018085cb7c031a6a2f1a51","created_at":"2026-08-27T02:08:38.886Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-907b88f0ef4a4477","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:38.894Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fa018085cb7c031a6a2f1a51: start","job_status":"start","queue_duration_s":1.616034,"scheduling_latency_s":1.616061} gitlab # [ 443.905752] gitaly[906]: time="2026-08-27T02:08:40.603Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:10.601" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:40.602" grpc.time_ms=1.423 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 443.919313] gitaly[906]: time="2026-08-27T02:08:40.626Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-08-27T02:09:10.615" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:40.616" grpc.time_ms=9.745 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 443.934410] gitaly[906]: time="2026-08-27T02:08:40.646Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:08:50.074" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:40.635" grpc.time_ms=11.546 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 443.948772] gitaly[906]: time="2026-08-27T02:08:40.651Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:40.650" grpc.time_ms=0.669 pid=906 span.kind=server system=grpc gitlab # [ 443.958497] gitaly[906]: time="2026-08-27T02:08:40.673Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-27T02:08:50.112" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:40.673" grpc.time_ms=0.333 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 443.970204] gitaly[906]: time="2026-08-27T02:08:40.683Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:40.681" grpc.time_ms=1.481 pid=906 span.kind=server system=grpc gitlab # [ 443.980502] gitaly[906]: time="2026-08-27T02:08:40.690Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-27T02:08:50.119" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:40.679" grpc.time_ms=10.674 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 443.994586] gitaly[906]: time="2026-08-27T02:08:40.724Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:40.723" grpc.time_ms=0.485 pid=906 span.kind=server system=grpc gitlab # [ 444.004939] gitaly[906]: time="2026-08-27T02:08:40.727Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:40.726" grpc.time_ms=0.496 pid=906 span.kind=server system=grpc gitlab # [ 444.015251] gitaly[906]: time="2026-08-27T02:08:40.730Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX diskcache=4234285d-7bd6-4580-aab1-da5415362df5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:10.610" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:40.610" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.025801] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.855Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fa018085cb7c031a6a2f1a51","created_at":"2026-08-27T02:08:38.886Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-907b88f0ef4a4477","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:38.894Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3569,"job_status":"rescheduled","message":"PipelineProcessWorker JID-fa018085cb7c031a6a2f1a51: rescheduled"} gitlab # [ 444.041934] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.865Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fa018085cb7c031a6a2f1a51","created_at":"2026-08-27T02:08:38.886Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-907b88f0ef4a4477","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:38.894Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fa018085cb7c031a6a2f1a51: done: 0.354405 sec","job_status":"done","queue_duration_s":1.616034,"scheduling_latency_s":1.616061,"gitaly_calls":3,"gitaly_duration_s":0.132786,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025945,"redis_read_bytes":148,"redis_write_bytes":18009,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.00309,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000898,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003134,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12308,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.005985,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00407,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001774,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003904,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.011,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.163301,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.302236,"duration_s":0.354405,"completed_at":"2026-08-27T02:08:40.865Z","load_balancing_strategy":"primary","db_duration_s":0.040581,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 444.098292] gitaly[906]: time="2026-08-27T02:08:40.730Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=112 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7096 command.oublock=0 command.real_time_ms=116 command.spawn_token_fork_ms=1 command.system_time_ms=39 command.user_time_ms=73 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:10.610" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:40.610" grpc.time_ms=119.932 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.111965] gitaly[906]: time="2026-08-27T02:08:40.761Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:40.760" grpc.time_ms=0.628 pid=906 span.kind=server system=grpc gitlab # [ 444.122000] nixbot[3632]: {"ts": "2026-08-27T02:08:40.913Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/029b33b836639f8d58dd1784db28a8cf810aca0d \"HTTP/1.1 201 Created\""} gitlab # [ 444.124842] nixbot[3632]: {"ts": "2026-08-27T02:08:40.924Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-029b33b83663-4"} gitlab # [ 444.127157] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.867Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","217dfa13f4ab76c8b701e53f"],"class":"WebHooks::LogExecutionWorker","jid":"c5177ec442679107a72e8f7b","created_at":"2026-08-27T02:08:39.475Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-081752923fd647e5","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1b2cdaeb1b05a06bc0e83282fc3f1a5cf981f08ca1b3402af8af394b7cfea0ab","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:39.482Z","job_size_bytes":2591,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c5177ec442679107a72e8f7b: start","job_status":"start","queue_duration_s":1.38488,"scheduling_latency_s":1.384911} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d' gitlab # [ 444.144702] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.915Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","217dfa13f4ab76c8b701e53f"],"class":"WebHooks::LogExecutionWorker","jid":"c5177ec442679107a72e8f7b","created_at":"2026-08-27T02:08:39.475Z","trace_propagation_headers":{"sentry-trace":"421795e5377d4c04afc7bb1ca3e48620-081752923fd647e5","baggage":"sentry-trace_id=421795e5377d4c04afc7bb1ca3e48620,sentry-release=Unknown"},"correlation_id":"01M10FH402T6CB0A36PR2ERK51","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1b2cdaeb1b05a06bc0e83282fc3f1a5cf981f08ca1b3402af8af394b7cfea0ab","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:39.482Z","job_size_bytes":2591,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c5177ec442679107a72e8f7b: done: 0.04833 sec","job_status":"done","queue_duration_s":1.38488,"scheduling_latency_s":1.384911,"redis_calls":7,"redis_duration_s":0.003555,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04833,"completed_at":"2026-08-27T02:08:40.915Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.189510] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 401 gitlab # [ 444.191820] gitaly[906]: time="2026-08-27T02:08:40.773Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 diskcache=9a2280c2-5ed3-45a1-a2f4-673fb010d55e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-27T02:08:50.133" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:40.693" pid=906 ref=refs/keep-around/029b33b836639f8d58dd1784db28a8cf810aca0d remote_ip="2001:db8:1::1" revision=029b33b836639f8d58dd1784db28a8cf810aca0d span.kind=server system=grpc username=root gitlab # [ 444.202788] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:40.918Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"d681c9a29d9d18a3a4a0475c","created_at":"2026-08-27T02:08:40.428Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-3119e74036614aef","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.434Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-d681c9a29d9d18a3a4a0475c: start","job_status":"start","queue_duration_s":0.483994,"scheduling_latency_s":0.484033} gitlab # [ 444.218815] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.059Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"d681c9a29d9d18a3a4a0475c","created_at":"2026-08-27T02:08:40.428Z","trace_propagation_headers":{"sentry-trace":"4d9e2c3fd98d454db15d91dcd91ca107-3119e74036614aef","baggage":"sentry-trace_id=4d9e2c3fd98d454db15d91dcd91ca107,sentry-release=Unknown"},"correlation_id":"01M10FGY7BS202X1T11VQVMM02","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.434Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-d681c9a29d9d18a3a4a0475c: done: 0.140835 sec","job_status":"done","queue_duration_s":0.483994,"scheduling_latency_s":0.484033,"redis_calls":11,"redis_duration_s":0.004464,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001628,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001898,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058158,"duration_s":0.140835,"completed_at":"2026-08-27T02:08:41.058Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.267447] gitaly[906]: time="2026-08-27T02:08:40.773Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5195 command.oublock=8 command.real_time_ms=74 command.spawn_token_fork_ms=0 command.system_time_ms=30 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-27T02:08:50.133" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-27T02:08:40.693" grpc.time_ms=79.622 pid=906 ref=refs/keep-around/029b33b836639f8d58dd1784db28a8cf810aca0d remote_ip="2001:db8:1::1" revision=029b33b836639f8d58dd1784db28a8cf810aca0d span.kind=server system=grpc username=root gitlab # [ 444.283151] nixbot[3632]: {"ts": "2026-08-27T02:08:41.126Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-029b33b83663-4", "argv0": "bwrap"} gitlab # [ 444.285742] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.061Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5636f3fcb99ecf9d13aaaadd","created_at":"2026-08-27T02:08:40.600Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.602Z","job_size_bytes":23,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5636f3fcb99ecf9d13aaaadd: start","job_status":"start","queue_duration_s":0.458113,"scheduling_latency_s":0.458148} gitlab # [ 444.302877] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.128Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5636f3fcb99ecf9d13aaaadd","created_at":"2026-08-27T02:08:40.600Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.602Z","job_size_bytes":23,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5636f3fcb99ecf9d13aaaadd: done: 0.067176 sec","job_status":"done","queue_duration_s":0.458113,"scheduling_latency_s":0.458148,"redis_calls":7,"redis_duration_s":0.001698,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000485,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067176,"completed_at":"2026-08-27T02:08:41.128Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008731,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 444.348091] gitaly[906]: time="2026-08-27T02:08:40.779Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:10.778" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:40.778" grpc.time_ms=0.449 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.361068] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.130Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"dcd29e4a7dd0e2557b06e5b2","created_at":"2026-08-27T02:08:40.739Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.742Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-dcd29e4a7dd0e2557b06e5b2: start","job_status":"start","queue_duration_s":0.387334,"scheduling_latency_s":0.387369} gitlab # [ 444.378194] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.174Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"dcd29e4a7dd0e2557b06e5b2","created_at":"2026-08-27T02:08:40.739Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.742Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-dcd29e4a7dd0e2557b06e5b2: done: 0.044919 sec","job_status":"done","queue_duration_s":0.387334,"scheduling_latency_s":0.387369,"redis_calls":7,"redis_duration_s":0.001574,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044919,"completed_at":"2026-08-27T02:08:41.174Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.422380] gitaly[906]: time="2026-08-27T02:08:40.805Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-27T02:09:10.804" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:40.804" grpc.time_ms=0.698 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.435510] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.176Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"33649c896f79333499d449cf","created_at":"2026-08-27T02:08:40.752Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.757Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-33649c896f79333499d449cf: start","job_status":"start","queue_duration_s":0.418839,"scheduling_latency_s":0.418875} gitlab # [ 444.451979] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.232Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"33649c896f79333499d449cf","created_at":"2026-08-27T02:08:40.752Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.757Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-33649c896f79333499d449cf: done: 0.055548 sec","job_status":"done","queue_duration_s":0.418839,"scheduling_latency_s":0.418875,"redis_calls":7,"redis_duration_s":0.002162,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001583,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000579,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055548,"completed_at":"2026-08-27T02:08:41.232Z","load_balancing_strategy":"primary","db_duration_s":0.005591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.495721] gitaly[906]: time="2026-08-27T02:08:40.843Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKXPBTR1KFNGRXF81N4XX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:10.842" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:40.843" grpc.time_ms=0.698 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.508745] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.236Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e1f20e9ad903098ac8dd6f1b","created_at":"2026-08-27T02:08:40.769Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.772Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e1f20e9ad903098ac8dd6f1b: start","job_status":"start","queue_duration_s":0.463583,"scheduling_latency_s":0.463642} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d', in 0.38 seconds) (finished: subtest: push delivers the webhook, in 95.02 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 444.525698] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.334Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e1f20e9ad903098ac8dd6f1b","created_at":"2026-08-27T02:08:40.769Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.772Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e1f20e9ad903098ac8dd6f1b: done: 0.098055 sec","job_status":"done","queue_duration_s":0.463583,"scheduling_latency_s":0.463642,"redis_calls":10,"redis_duration_s":0.006518,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003674,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000999,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001845,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.098055,"completed_at":"2026-08-27T02:08:41.333Z","load_balancing_strategy":"primary","db_duration_s":0.009167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.570970] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.340Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e3f93920cb479aa4844dff91","created_at":"2026-08-27T02:08:40.773Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.775Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e3f93920cb479aa4844dff91: start","job_status":"start","queue_duration_s":0.565108,"scheduling_latency_s":0.565161} gitlab # [ 444.588513] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.409Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e3f93920cb479aa4844dff91","created_at":"2026-08-27T02:08:40.773Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.775Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e3f93920cb479aa4844dff91: done: 0.069189 sec","job_status":"done","queue_duration_s":0.565108,"scheduling_latency_s":0.565161,"redis_calls":7,"redis_duration_s":0.0033559999999999996,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000861,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069189,"completed_at":"2026-08-27T02:08:41.409Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.634083] nixbot[3632]: INFO: ::ffff:127.0.0.1:44892 - "GET /api/repos/gitlab/root/test-flake/builds?commit=029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 200 OK gitlab # [ 444.636186] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.418Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"fe8eeb5febd7fd97674ce69f","created_at":"2026-08-27T02:08:40.776Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.778Z","job_size_bytes":19,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-fe8eeb5febd7fd97674ce69f: start","job_status":"start","queue_duration_s":0.639226,"scheduling_latency_s":0.639274} gitlab # [ 444.653506] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.477Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"fe8eeb5febd7fd97674ce69f","created_at":"2026-08-27T02:08:40.776Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.778Z","job_size_bytes":19,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-fe8eeb5febd7fd97674ce69f: done: 0.059427 sec","job_status":"done","queue_duration_s":0.639226,"scheduling_latency_s":0.639274,"redis_calls":8,"redis_duration_s":0.00771,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00678,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059427,"completed_at":"2026-08-27T02:08:41.477Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.699630] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.481Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d24b367f3e8e41606f6bcf0e","created_at":"2026-08-27T02:08:40.780Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.782Z","job_size_bytes":66,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d24b367f3e8e41606f6bcf0e: start","job_status":"start","queue_duration_s":0.69862,"scheduling_latency_s":0.698659} gitlab # [ 444.718366] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.523Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d24b367f3e8e41606f6bcf0e","created_at":"2026-08-27T02:08:40.780Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.782Z","job_size_bytes":66,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d24b367f3e8e41606f6bcf0e: done: 0.042236 sec","job_status":"done","queue_duration_s":0.69862,"scheduling_latency_s":0.698659,"redis_calls":7,"redis_duration_s":0.004092,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002805,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001287,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042236,"completed_at":"2026-08-27T02:08:41.523Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.763458] nix-daemon[3863]: accepted connection from pid 4517, user nixbot gitlab # [ 444.764527] gitaly[906]: time="2026-08-27T02:08:41.601Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FM0GSCHBW8KE99MXT3WVS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:11.599" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:41.600" grpc.time_ms=0.779 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.777628] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.527Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"82e33e364ceee51e12e8d348","created_at":"2026-08-27T02:08:40.783Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.785Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-82e33e364ceee51e12e8d348: start","job_status":"start","queue_duration_s":0.741542,"scheduling_latency_s":0.741576} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.25 seconds) gitlab # [ 444.794606] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.570Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"82e33e364ceee51e12e8d348","created_at":"2026-08-27T02:08:40.783Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.785Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-82e33e364ceee51e12e8d348: done: 0.04318 sec","job_status":"done","queue_duration_s":0.741542,"scheduling_latency_s":0.741576,"redis_calls":7,"redis_duration_s":0.004347,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002202,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04318,"completed_at":"2026-08-27T02:08:41.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.837419] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.572Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"6ff9ed9cdb0832efe97fcc34","created_at":"2026-08-27T02:08:40.781Z","trace_propagation_headers":{"sentry-trace":"bee2974104b3468da9ed7ea756eb947c-b775f88429e3434a","baggage":"sentry-trace_id=bee2974104b3468da9ed7ea756eb947c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKZHZA06FAJ3K8GQQKMN5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.788Z","job_size_bytes":65,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-6ff9ed9cdb0832efe97fcc34: start","job_status":"start","queue_duration_s":0.784406,"scheduling_latency_s":0.784434} gitlab # [ 444.854645] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.624Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"6ff9ed9cdb0832efe97fcc34","created_at":"2026-08-27T02:08:40.781Z","trace_propagation_headers":{"sentry-trace":"bee2974104b3468da9ed7ea756eb947c-b775f88429e3434a","baggage":"sentry-trace_id=bee2974104b3468da9ed7ea756eb947c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKZHZA06FAJ3K8GQQKMN5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.788Z","job_size_bytes":65,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-6ff9ed9cdb0832efe97fcc34: done: 0.051523 sec","job_status":"done","queue_duration_s":0.784406,"scheduling_latency_s":0.784434,"redis_calls":7,"redis_duration_s":0.006425999999999999,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005957,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051523,"completed_at":"2026-08-27T02:08:41.624Z","load_balancing_strategy":"primary","db_duration_s":0.010636,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 444.899013] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:41 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 172 gitlab # [ 444.901826] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.626Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"5bd32e3501d0afd9b3f38302","created_at":"2026-08-27T02:08:40.786Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.789Z","job_size_bytes":6,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-5bd32e3501d0afd9b3f38302: start","job_status":"start","queue_duration_s":0.837408,"scheduling_latency_s":0.83744} gitlab # [ 444.918126] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.651Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"5bd32e3501d0afd9b3f38302","created_at":"2026-08-27T02:08:40.786Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.789Z","job_size_bytes":6,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-5bd32e3501d0afd9b3f38302: done: 0.024704 sec","job_status":"done","queue_duration_s":0.837408,"scheduling_latency_s":0.83744,"redis_calls":7,"redis_duration_s":0.004221,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001461,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024704,"completed_at":"2026-08-27T02:08:41.651Z","load_balancing_strategy":"primary","db_duration_s":0.000755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 444.962283] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.653Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"35cd9346826fdd2eaec1bbf4","created_at":"2026-08-27T02:08:40.789Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.791Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-35cd9346826fdd2eaec1bbf4: start","job_status":"start","queue_duration_s":0.861634,"scheduling_latency_s":0.861662} gitlab # [ 444.978180] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.692Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"35cd9346826fdd2eaec1bbf4","created_at":"2026-08-27T02:08:40.789Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.791Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-35cd9346826fdd2eaec1bbf4: done: 0.039139 sec","job_status":"done","queue_duration_s":0.861634,"scheduling_latency_s":0.861662,"redis_calls":7,"redis_duration_s":0.002193,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001569,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000624,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039139,"completed_at":"2026-08-27T02:08:41.692Z","load_balancing_strategy":"primary","db_duration_s":0.009162,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 445.020834] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.694Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8e3c1ca745602464f9b87a73","created_at":"2026-08-27T02:08:40.789Z","trace_propagation_headers":{"sentry-trace":"bee2974104b3468da9ed7ea756eb947c-b775f88429e3434a","baggage":"sentry-trace_id=bee2974104b3468da9ed7ea756eb947c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKZHZA06FAJ3K8GQQKMN5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.794Z","job_size_bytes":65,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8e3c1ca745602464f9b87a73: start","job_status":"start","queue_duration_s":0.900461,"scheduling_latency_s":0.900489} gitlab # [ 445.038716] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.777Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8e3c1ca745602464f9b87a73","created_at":"2026-08-27T02:08:40.789Z","trace_propagation_headers":{"sentry-trace":"bee2974104b3468da9ed7ea756eb947c-b775f88429e3434a","baggage":"sentry-trace_id=bee2974104b3468da9ed7ea756eb947c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKZHZA06FAJ3K8GQQKMN5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.794Z","job_size_bytes":65,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8e3c1ca745602464f9b87a73: done: 0.083025 sec","job_status":"done","queue_duration_s":0.900461,"scheduling_latency_s":0.900489,"redis_calls":10,"redis_duration_s":0.004165,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002419,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001746,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.083025,"completed_at":"2026-08-27T02:08:41.777Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 445.083979] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.780Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4d91ec3186c2368a52b0e923","created_at":"2026-08-27T02:08:40.856Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.858Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4d91ec3186c2368a52b0e923: start","job_status":"start","queue_duration_s":0.921402,"scheduling_latency_s":0.921431} gitlab # [ 445.100406] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.819Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4d91ec3186c2368a52b0e923","created_at":"2026-08-27T02:08:40.856Z","trace_propagation_headers":{"sentry-trace":"00157cf820304f5eaa0ab6791f525415-6992e92703d1474d","baggage":"sentry-trace_id=00157cf820304f5eaa0ab6791f525415,sentry-release=Unknown"},"correlation_id":"01M10FKXPBTR1KFNGRXF81N4XX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.858Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4d91ec3186c2368a52b0e923: done: 0.039053 sec","job_status":"done","queue_duration_s":0.921402,"scheduling_latency_s":0.921431,"redis_calls":8,"redis_duration_s":0.002492,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00196,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039053,"completed_at":"2026-08-27T02:08:41.818Z","load_balancing_strategy":"primary","db_duration_s":0.006754,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 445.144272] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:41.821Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"227201886f50355c927e3ffd","created_at":"2026-08-27T02:08:40.882Z","trace_propagation_headers":{"sentry-trace":"bee2974104b3468da9ed7ea756eb947c-b775f88429e3434a","baggage":"sentry-trace_id=bee2974104b3468da9ed7ea756eb947c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKZHZA06FAJ3K8GQQKMN5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.886Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-227201886f50355c927e3ffd: start","job_status":"start","queue_duration_s":0.934047,"scheduling_latency_s":0.934073} gitlab # [ 445.160498] gitaly[906]: time="2026-08-27T02:08:41.977Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-27T02:09:11.075" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:41.975" grpc.time_ms=1.841 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 445.194269] gitaly[906]: time="2026-08-27T02:08:42.065Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FKZHZA06FAJ3K8GQQKMN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:12.063" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:42.064" grpc.time_ms=1.275 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 445.225042] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:42.093Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"227201886f50355c927e3ffd","created_at":"2026-08-27T02:08:40.882Z","trace_propagation_headers":{"sentry-trace":"bee2974104b3468da9ed7ea756eb947c-b775f88429e3434a","baggage":"sentry-trace_id=bee2974104b3468da9ed7ea756eb947c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FKZHZA06FAJ3K8GQQKMN5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:40.886Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-227201886f50355c927e3ffd: done: 0.272257 sec","job_status":"done","queue_duration_s":0.934047,"scheduling_latency_s":0.934073,"gitaly_calls":2,"gitaly_duration_s":0.012476,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013426,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002101,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002446,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000591,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002917,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000605,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000925,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00174,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.219279,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.2180299,"duration_s":0.272257,"completed_at":"2026-08-27T02:08:42.093Z","load_balancing_strategy":"primary","db_duration_s":0.045526,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 445.296362] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:42.097Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"32aad1d4e52b34306a438e73","created_at":"2026-08-27T02:08:41.941Z","trace_propagation_headers":{"sentry-trace":"bee2974104b3468da9ed7ea756eb947c-a03c4b3d145744b8","baggage":"sentry-trace_id=bee2974104b3468da9ed7ea756eb947c,sentry-release=Unknown"},"correlation_id":"01M10FKZHZA06FAJ3K8GQQKMN5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:41.946Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-32aad1d4e52b34306a438e73: start","job_status":"start","queue_duration_s":0.150505,"scheduling_latency_s":0.150556} gitlab # [ 445.314212] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:42.177Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"32aad1d4e52b34306a438e73","created_at":"2026-08-27T02:08:41.941Z","trace_propagation_headers":{"sentry-trace":"bee2974104b3468da9ed7ea756eb947c-a03c4b3d145744b8","baggage":"sentry-trace_id=bee2974104b3468da9ed7ea756eb947c,sentry-release=Unknown"},"correlation_id":"01M10FKZHZA06FAJ3K8GQQKMN5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:41.946Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-32aad1d4e52b34306a438e73: done: 0.080397 sec","job_status":"done","queue_duration_s":0.150505,"scheduling_latency_s":0.150556,"redis_calls":7,"redis_duration_s":0.003066,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.080397,"completed_at":"2026-08-27T02:08:42.177Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010174,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 446.017127] gitaly[906]: time="2026-08-27T02:08:42.887Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FM1R8KDWSKMC3R4FBS9DN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:12.884" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:42.885" grpc.time_ms=1.406 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.28 seconds) gitlab # [ 446.086628] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 196 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 447.072771] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:43.941Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ea4fb1232ee800b6cca0d434","created_at":"2026-08-27T02:08:43.918Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83da9bcc3db5f2cb9d79cbaa61b2fdde","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:43.935Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ea4fb1232ee800b6cca0d434: start","job_status":"start","queue_duration_s":0.005968,"scheduling_latency_s":0.006033} gitlab # [ 447.223172] gitaly[906]: time="2026-08-27T02:08:44.093Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FM2ZBXTHQ84VREGD350G4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:14.091" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:44.092" grpc.time_ms=1.2 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.20 seconds) gitlab # [ 447.288344] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 146 gitlab # [ 447.323231] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:44.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ea4fb1232ee800b6cca0d434","created_at":"2026-08-27T02:08:43.918Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83da9bcc3db5f2cb9d79cbaa61b2fdde","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:43.935Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ea4fb1232ee800b6cca0d434: done: 0.252017 sec","job_status":"done","queue_duration_s":0.005968,"scheduling_latency_s":0.006033,"redis_calls":9,"redis_duration_s":0.033009,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.014888,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004399,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.252017,"completed_at":"2026-08-27T02:08:44.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.13387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 447.411258] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:44.208Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3c789bc58d5d25f3a35dbae0","created_at":"2026-08-27T02:08:43.964Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.001Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3c789bc58d5d25f3a35dbae0: start","job_status":"start","queue_duration_s":0.206659,"scheduling_latency_s":0.206718} gitlab # [ 448.140749] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3c789bc58d5d25f3a35dbae0","created_at":"2026-08-27T02:08:43.964Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.001Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3c789bc58d5d25f3a35dbae0: done: 0.801624 sec","job_status":"done","queue_duration_s":0.206659,"scheduling_latency_s":0.206718,"redis_calls":70,"redis_duration_s":0.30807300000000004,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.022754,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00182,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.277056,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006443,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.333784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.801624,"completed_at":"2026-08-27T02:08:45.009Z","load_balancing_strategy":"primary","db_duration_s":0.038129,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 448.152488] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.023Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"60265047fd5aa6bcd20d4ea4","created_at":"2026-08-27T02:08:44.033Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"182c8967ece970b092ca82db50f6838b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.062Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-60265047fd5aa6bcd20d4ea4: start","job_status":"start","queue_duration_s":0.960988,"scheduling_latency_s":0.961047} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 448.342676] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"60265047fd5aa6bcd20d4ea4","created_at":"2026-08-27T02:08:44.033Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"182c8967ece970b092ca82db50f6838b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.062Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-60265047fd5aa6bcd20d4ea4: done: 0.189939 sec","job_status":"done","queue_duration_s":0.960988,"scheduling_latency_s":0.961047,"redis_calls":11,"redis_duration_s":0.034802,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006164,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.021486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007152,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002406,"exclusive_lock_hold_duration_s":0.03112538,"duration_s":0.189939,"completed_at":"2026-08-27T02:08:45.212Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 448.429416] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"851686699bc5f21c7fd5df07","created_at":"2026-08-27T02:08:44.070Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62ab53e71d26856819d202ae07de5df4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.089Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-851686699bc5f21c7fd5df07: start","job_status":"start","queue_duration_s":1.129815,"scheduling_latency_s":1.129871} gitlab # [ 448.464578] gitaly[906]: time="2026-08-27T02:08:45.335Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FM452FXS7QQP7A5YW869P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:15.333" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:45.334" grpc.time_ms=1.038 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 448.528111] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 179 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.24 seconds) gitlab # [ 448.567859] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.438Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"851686699bc5f21c7fd5df07","created_at":"2026-08-27T02:08:44.070Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62ab53e71d26856819d202ae07de5df4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.089Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-851686699bc5f21c7fd5df07: done: 0.219511 sec","job_status":"done","queue_duration_s":1.129815,"scheduling_latency_s":1.129871,"redis_calls":11,"redis_duration_s":0.0682,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004673,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.056077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00745,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00182284,"exclusive_lock_hold_duration_s":0.00830774,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.219511,"completed_at":"2026-08-27T02:08:45.438Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 448.640372] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"72a25a7be545adf40ccde881","created_at":"2026-08-27T02:08:44.105Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e79f6817a26dd40bd3f366e147bf3c21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-27T02:08:44.142Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-72a25a7be545adf40ccde881: start","job_status":"start","queue_duration_s":1.299222,"scheduling_latency_s":1.299269} gitlab # [ 448.656485] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.470Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 448.660908] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"72a25a7be545adf40ccde881","created_at":"2026-08-27T02:08:44.105Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e79f6817a26dd40bd3f366e147bf3c21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-27T02:08:44.142Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-72a25a7be545adf40ccde881: done: 0.047906 sec","job_status":"done","queue_duration_s":1.299222,"scheduling_latency_s":1.299269,"redis_calls":9,"redis_duration_s":0.004317,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001024,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002458,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000835,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047906,"completed_at":"2026-08-27T02:08:45.489Z","load_balancing_strategy":"primary","db_duration_s":0.001179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 448.707346] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"95b0397fb7fcde31f0e4f607","created_at":"2026-08-27T02:08:44.152Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88ad803b26c28776ccc46a6a343660e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.187Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-95b0397fb7fcde31f0e4f607: start","job_status":"start","queue_duration_s":1.306204,"scheduling_latency_s":1.306258} gitlab # [ 448.756421] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"95b0397fb7fcde31f0e4f607","created_at":"2026-08-27T02:08:44.152Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88ad803b26c28776ccc46a6a343660e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.187Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-95b0397fb7fcde31f0e4f607: done: 0.13351 sec","job_status":"done","queue_duration_s":1.306204,"scheduling_latency_s":1.306258,"redis_calls":13,"redis_duration_s":0.005469,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001466,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001896,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.024,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00129076,"exclusive_lock_hold_duration_s":0.07973248,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.13351,"completed_at":"2026-08-27T02:08:45.627Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028103,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 448.808383] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dc8586c7cf27d1defbb66d47","created_at":"2026-08-27T02:08:44.207Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42431f3b8d41288869f206f7d508ebd0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.228Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dc8586c7cf27d1defbb66d47: start","job_status":"start","queue_duration_s":1.401721,"scheduling_latency_s":1.401764} gitlab # [ 448.823155] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dc8586c7cf27d1defbb66d47","created_at":"2026-08-27T02:08:44.207Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42431f3b8d41288869f206f7d508ebd0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.228Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dc8586c7cf27d1defbb66d47: done: 0.045568 sec","job_status":"done","queue_duration_s":1.401721,"scheduling_latency_s":1.401764,"redis_calls":9,"redis_duration_s":0.003191,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000669,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001829,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000693,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045568,"completed_at":"2026-08-27T02:08:45.675Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 448.868630] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2e8aaa374f942d9ef73610ca","created_at":"2026-08-27T02:08:44.238Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bb747d8955a112b773f8386771d483c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-27T02:08:44.268Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2e8aaa374f942d9ef73610ca: start","job_status":"start","queue_duration_s":1.409726,"scheduling_latency_s":1.409761} gitlab # [ 448.909113] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.731Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2e8aaa374f942d9ef73610ca","created_at":"2026-08-27T02:08:44.238Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bb747d8955a112b773f8386771d483c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-27T02:08:44.268Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2e8aaa374f942d9ef73610ca: done: 0.052666 sec","job_status":"done","queue_duration_s":1.409726,"scheduling_latency_s":1.409761,"redis_calls":9,"redis_duration_s":0.003171,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00067,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001828,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000673,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052666,"completed_at":"2026-08-27T02:08:45.730Z","load_balancing_strategy":"primary","db_duration_s":0.007598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 448.988616] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.733Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"368f420c87474c79d6ebc60b","created_at":"2026-08-27T02:08:44.292Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"229e4e2212c520fc1bdb768eadd2a478","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.320Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-368f420c87474c79d6ebc60b: start","job_status":"start","queue_duration_s":1.413375,"scheduling_latency_s":1.413412} gitlab # [ 449.028706] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.775Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"368f420c87474c79d6ebc60b","created_at":"2026-08-27T02:08:44.292Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"229e4e2212c520fc1bdb768eadd2a478","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.320Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-368f420c87474c79d6ebc60b: done: 0.041581 sec","job_status":"done","queue_duration_s":1.413375,"scheduling_latency_s":1.413412,"redis_calls":11,"redis_duration_s":0.003517,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000687,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001391,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008835,"exclusive_lock_hold_duration_s":0.01158772,"duration_s":0.041581,"completed_at":"2026-08-27T02:08:45.775Z","load_balancing_strategy":"primary","db_duration_s":0.005893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 449.104646] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"429a20757c27c01f1bf7bdcb","created_at":"2026-08-27T02:08:44.346Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.372Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-429a20757c27c01f1bf7bdcb: start","job_status":"start","queue_duration_s":1.404358,"scheduling_latency_s":1.404388} gitlab # [ 449.121906] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"429a20757c27c01f1bf7bdcb","created_at":"2026-08-27T02:08:44.346Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.372Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-429a20757c27c01f1bf7bdcb: done: 0.048052 sec","job_status":"done","queue_duration_s":1.404358,"scheduling_latency_s":1.404388,"redis_calls":18,"redis_duration_s":0.0043879999999999995,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000967,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000495,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002084,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000842,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048052,"completed_at":"2026-08-27T02:08:45.824Z","load_balancing_strategy":"primary","db_duration_s":0.003664,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 449.167953] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.826Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"936f033e7499f6086392b4ac","created_at":"2026-08-27T02:08:44.416Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f6facded6e550bf5e4452e8c45fe536","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.439Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-936f033e7499f6086392b4ac: start","job_status":"start","queue_duration_s":1.387306,"scheduling_latency_s":1.387335} gitlab # [ 449.181920] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.850Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"936f033e7499f6086392b4ac","created_at":"2026-08-27T02:08:44.416Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f6facded6e550bf5e4452e8c45fe536","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.439Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-936f033e7499f6086392b4ac: done: 0.023735 sec","job_status":"done","queue_duration_s":1.387306,"scheduling_latency_s":1.387335,"redis_calls":11,"redis_duration_s":0.002842,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000504,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00053,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023735,"completed_at":"2026-08-27T02:08:45.850Z","load_balancing_strategy":"primary","db_duration_s":0.000742,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 449.225384] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.852Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9ad5c168c6d7dac06279c3ed","created_at":"2026-08-27T02:08:44.499Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d3c5368a1eb65ceff9ffd24d37608d4","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.536Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9ad5c168c6d7dac06279c3ed: start","job_status":"start","queue_duration_s":1.315727,"scheduling_latency_s":1.315758} gitlab # [ 449.240150] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.892Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9ad5c168c6d7dac06279c3ed","created_at":"2026-08-27T02:08:44.499Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d3c5368a1eb65ceff9ffd24d37608d4","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.536Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9ad5c168c6d7dac06279c3ed: done: 0.040008 sec","job_status":"done","queue_duration_s":1.315727,"scheduling_latency_s":1.315758,"redis_calls":12,"redis_duration_s":0.003581,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001717,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001238,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01203232,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.040008,"completed_at":"2026-08-27T02:08:45.892Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004589,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 449.286194] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:45.895Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"725fd2ffe3dea74f597a7ea9","created_at":"2026-08-27T02:08:44.571Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c230353dc3df1421ba7d45a600f7258","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.601Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-725fd2ffe3dea74f597a7ea9: start","job_status":"start","queue_duration_s":1.293573,"scheduling_latency_s":1.293602} gitlab # [ 449.463114] nix-daemon[3863]: accepted connection from pid 4557, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 449.522122] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"725fd2ffe3dea74f597a7ea9","created_at":"2026-08-27T02:08:44.571Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c230353dc3df1421ba7d45a600f7258","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.601Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-725fd2ffe3dea74f597a7ea9: done: 0.497853 sec","job_status":"done","queue_duration_s":1.293573,"scheduling_latency_s":1.293602,"redis_calls":564,"redis_duration_s":0.12317,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000596,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.121041,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.42503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.497853,"completed_at":"2026-08-27T02:08:46.392Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000799,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 449.633489] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"9a3613925b6a97ce512e4aed","created_at":"2026-08-27T02:08:44.671Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bde5c59835c1fcf281d5e2510f52b6c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.706Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-9a3613925b6a97ce512e4aed: start","job_status":"start","queue_duration_s":1.688781,"scheduling_latency_s":1.688824} gitlab # [ 449.655142] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.463Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"9a3613925b6a97ce512e4aed","created_at":"2026-08-27T02:08:44.671Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bde5c59835c1fcf281d5e2510f52b6c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.706Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-9a3613925b6a97ce512e4aed: done: 0.068025 sec","job_status":"done","queue_duration_s":1.688781,"scheduling_latency_s":1.688824,"redis_calls":11,"redis_duration_s":0.00287,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00104508,"exclusive_lock_hold_duration_s":0.02214308,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.068025,"completed_at":"2026-08-27T02:08:46.463Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 449.706769] nix-daemon[3863]: accepted connection from pid 4564, user nixbot gitlab # [ 449.707769] nix-daemon[3863]: accepted connection from pid 4517, user nixbot gitlab # [ 449.708829] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4d8cf97ba05564bc31e1b07b","created_at":"2026-08-27T02:08:44.729Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f634d8338b7f14c48812e2546389dabc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.765Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4d8cf97ba05564bc31e1b07b: start","job_status":"start","queue_duration_s":1.701434,"scheduling_latency_s":1.701467} gitlab # [ 449.724403] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.484Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 449.728074] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.496Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4d8cf97ba05564bc31e1b07b","created_at":"2026-08-27T02:08:44.729Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f634d8338b7f14c48812e2546389dabc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.765Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4d8cf97ba05564bc31e1b07b: done: 0.029385 sec","job_status":"done","queue_duration_s":1.701434,"scheduling_latency_s":1.701467,"redis_calls":9,"redis_duration_s":0.001889,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000394,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029385,"completed_at":"2026-08-27T02:08:46.496Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 449.772601] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"fba49a4c7657b8c0e925fdec","created_at":"2026-08-27T02:08:44.810Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8cb2ebbf32b8585e2928c3816d3693a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.839Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-fba49a4c7657b8c0e925fdec: start","job_status":"start","queue_duration_s":1.658783,"scheduling_latency_s":1.65881} gitlab # [ 449.787392] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"fba49a4c7657b8c0e925fdec","created_at":"2026-08-27T02:08:44.810Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8cb2ebbf32b8585e2928c3816d3693a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.839Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-fba49a4c7657b8c0e925fdec: done: 0.050771 sec","job_status":"done","queue_duration_s":1.658783,"scheduling_latency_s":1.65881,"redis_calls":9,"redis_duration_s":0.002212,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000469,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.050771,"completed_at":"2026-08-27T02:08:46.548Z","load_balancing_strategy":"primary","db_duration_s":0.017166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 449.832100] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.550Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"aac4018212efa5a362fdb232","created_at":"2026-08-27T02:08:44.876Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"944e9f0ec440b6e8aec00f313f67eed0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-27T02:08:44.904Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-aac4018212efa5a362fdb232: start","job_status":"start","queue_duration_s":1.645631,"scheduling_latency_s":1.645653} gitlab # [ 449.845771] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.563Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 449.849193] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.571Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"aac4018212efa5a362fdb232","created_at":"2026-08-27T02:08:44.876Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"944e9f0ec440b6e8aec00f313f67eed0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-27T02:08:44.904Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-aac4018212efa5a362fdb232: done: 0.020991 sec","job_status":"done","queue_duration_s":1.645631,"scheduling_latency_s":1.645653,"redis_calls":9,"redis_duration_s":0.001419,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020991,"completed_at":"2026-08-27T02:08:46.571Z","load_balancing_strategy":"primary","db_duration_s":0.000787,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 449.892654] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.572Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"3be3213b29c6009880b36d80","created_at":"2026-08-27T02:08:44.932Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"e7798bab45a66a1a7298014a56d56d9b","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.960Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-3be3213b29c6009880b36d80: start","job_status":"start","queue_duration_s":1.612266,"scheduling_latency_s":1.612288} gitlab # [ 449.907515] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.739Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"3be3213b29c6009880b36d80","created_at":"2026-08-27T02:08:44.932Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"e7798bab45a66a1a7298014a56d56d9b","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.960Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-3be3213b29c6009880b36d80: done: 0.166742 sec","job_status":"done","queue_duration_s":1.612266,"scheduling_latency_s":1.612288,"redis_calls":10,"redis_duration_s":0.001951,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.154025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036448,"duration_s":0.166742,"completed_at":"2026-08-27T02:08:46.739Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.43 seconds) gitlab # [ 449.953029] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d8fa8cfe9b6d748680a3a1f","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d8fa8cfe9b6d748680a3a1f: start","job_status":"start","queue_duration_s":1.774725,"scheduling_latency_s":1.774748} gitlab # [ 449.965690] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d8fa8cfe9b6d748680a3a1f","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d8fa8cfe9b6d748680a3a1f: done: 0.029679 sec","job_status":"done","queue_duration_s":1.774725,"scheduling_latency_s":1.774748,"redis_calls":15,"redis_duration_s":0.002685,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001519,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029679,"completed_at":"2026-08-27T02:08:46.770Z","load_balancing_strategy":"primary","db_duration_s":0.000897,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.008381] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"001eeec32f141a790b7955f0","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-001eeec32f141a790b7955f0: start","job_status":"start","queue_duration_s":1.81334,"scheduling_latency_s":1.813368} gitlab # [ 450.021467] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 307 gitlab # [ 450.024493] gitaly[906]: time="2026-08-27T02:08:46.771Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FM5DCJPW8YPCFHKQ8MCDF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:16.769" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:46.770" grpc.time_ms=0.662 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 450.037537] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"001eeec32f141a790b7955f0","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-001eeec32f141a790b7955f0: done: 0.041888 sec","job_status":"done","queue_duration_s":1.81334,"scheduling_latency_s":1.813368,"redis_calls":13,"redis_duration_s":0.013198999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.011132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041888,"completed_at":"2026-08-27T02:08:46.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.078069] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d569fe4775a60967d5ef6fa3","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d569fe4775a60967d5ef6fa3: start","job_status":"start","queue_duration_s":1.859256,"scheduling_latency_s":1.859288} gitlab # [ 450.091921] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d569fe4775a60967d5ef6fa3","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d569fe4775a60967d5ef6fa3: done: 0.023158 sec","job_status":"done","queue_duration_s":1.859256,"scheduling_latency_s":1.859288,"redis_calls":13,"redis_duration_s":0.0034539999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023158,"completed_at":"2026-08-27T02:08:46.848Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.133017] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45d009a4b0f9eba54ff90b20","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45d009a4b0f9eba54ff90b20: start","job_status":"start","queue_duration_s":1.885561,"scheduling_latency_s":1.885588} gitlab # [ 450.145756] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45d009a4b0f9eba54ff90b20","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45d009a4b0f9eba54ff90b20: done: 0.026956 sec","job_status":"done","queue_duration_s":1.885561,"scheduling_latency_s":1.885588,"redis_calls":13,"redis_duration_s":0.004818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001389,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003429,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026956,"completed_at":"2026-08-27T02:08:46.878Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.186597] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fd18d7f19b49c6a15cfa0b2","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fd18d7f19b49c6a15cfa0b2: start","job_status":"start","queue_duration_s":1.914992,"scheduling_latency_s":1.915025} gitlab # [ 450.199530] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fd18d7f19b49c6a15cfa0b2","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fd18d7f19b49c6a15cfa0b2: done: 0.024753 sec","job_status":"done","queue_duration_s":1.914992,"scheduling_latency_s":1.915025,"redis_calls":13,"redis_duration_s":0.0034130000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024753,"completed_at":"2026-08-27T02:08:46.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.240543] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd9f95cf8ff73e89c3d5bebb","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd9f95cf8ff73e89c3d5bebb: start","job_status":"start","queue_duration_s":1.941793,"scheduling_latency_s":1.94182} gitlab # [ 450.253220] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd9f95cf8ff73e89c3d5bebb","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd9f95cf8ff73e89c3d5bebb: done: 0.023174 sec","job_status":"done","queue_duration_s":1.941793,"scheduling_latency_s":1.94182,"redis_calls":13,"redis_duration_s":0.003047,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023174,"completed_at":"2026-08-27T02:08:46.931Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.294434] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bcc1d61e4e2b2593782b8e5","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bcc1d61e4e2b2593782b8e5: start","job_status":"start","queue_duration_s":1.966912,"scheduling_latency_s":1.966935} gitlab # [ 450.307322] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bcc1d61e4e2b2593782b8e5","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bcc1d61e4e2b2593782b8e5: done: 0.021726 sec","job_status":"done","queue_duration_s":1.966912,"scheduling_latency_s":1.966935,"redis_calls":13,"redis_duration_s":0.003788,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002696,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021726,"completed_at":"2026-08-27T02:08:46.955Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.348029] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5abc91842bbfc426fee3783c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5abc91842bbfc426fee3783c: start","job_status":"start","queue_duration_s":1.99045,"scheduling_latency_s":1.990471} gitlab # [ 450.360689] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5abc91842bbfc426fee3783c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5abc91842bbfc426fee3783c: done: 0.0206 sec","job_status":"done","queue_duration_s":1.99045,"scheduling_latency_s":1.990471,"redis_calls":13,"redis_duration_s":0.002853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000992,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0206,"completed_at":"2026-08-27T02:08:46.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.400676] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"230c6ccc8d866c7fceaa274c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-230c6ccc8d866c7fceaa274c: start","job_status":"start","queue_duration_s":2.012749,"scheduling_latency_s":2.01277} gitlab # [ 450.413307] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"230c6ccc8d866c7fceaa274c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-230c6ccc8d866c7fceaa274c: done: 0.018069 sec","job_status":"done","queue_duration_s":2.012749,"scheduling_latency_s":2.01277,"redis_calls":13,"redis_duration_s":0.002775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001763,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018069,"completed_at":"2026-08-27T02:08:46.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.453483] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:46.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45471ce0fbb767eec39b401d","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45471ce0fbb767eec39b401d: start","job_status":"start","queue_duration_s":2.032369,"scheduling_latency_s":2.032386} gitlab # [ 450.466294] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45471ce0fbb767eec39b401d","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45471ce0fbb767eec39b401d: done: 0.018903 sec","job_status":"done","queue_duration_s":2.032369,"scheduling_latency_s":2.032386,"redis_calls":13,"redis_duration_s":0.003388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00149,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018903,"completed_at":"2026-08-27T02:08:47.017Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.507658] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4212b2d6e82113813d4b254c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4212b2d6e82113813d4b254c: start","job_status":"start","queue_duration_s":2.052858,"scheduling_latency_s":2.052875} gitlab # [ 450.520624] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4212b2d6e82113813d4b254c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4212b2d6e82113813d4b254c: done: 0.017245 sec","job_status":"done","queue_duration_s":2.052858,"scheduling_latency_s":2.052875,"redis_calls":13,"redis_duration_s":0.002951,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017245,"completed_at":"2026-08-27T02:08:47.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.561073] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72a3e9fdc93b82820b4c2e29","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72a3e9fdc93b82820b4c2e29: start","job_status":"start","queue_duration_s":2.071647,"scheduling_latency_s":2.071663} gitlab # [ 450.573823] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72a3e9fdc93b82820b4c2e29","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72a3e9fdc93b82820b4c2e29: done: 0.01682 sec","job_status":"done","queue_duration_s":2.071647,"scheduling_latency_s":2.071663,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01682,"completed_at":"2026-08-27T02:08:47.054Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.614370] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"806d00d0d79cbb371ab25315","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-806d00d0d79cbb371ab25315: start","job_status":"start","queue_duration_s":2.090931,"scheduling_latency_s":2.090951} gitlab # [ 450.627326] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"806d00d0d79cbb371ab25315","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-806d00d0d79cbb371ab25315: done: 0.016481 sec","job_status":"done","queue_duration_s":2.090931,"scheduling_latency_s":2.090951,"redis_calls":13,"redis_duration_s":0.002732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001792,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016481,"completed_at":"2026-08-27T02:08:47.073Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.667956] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76d740012dd64267011dc742","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76d740012dd64267011dc742: start","job_status":"start","queue_duration_s":2.108969,"scheduling_latency_s":2.108985} gitlab # [ 450.680924] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76d740012dd64267011dc742","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76d740012dd64267011dc742: done: 0.016849 sec","job_status":"done","queue_duration_s":2.108969,"scheduling_latency_s":2.108985,"redis_calls":13,"redis_duration_s":0.002921,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016849,"completed_at":"2026-08-27T02:08:47.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.722252] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a86df3d92a8bb7355cdff67","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a86df3d92a8bb7355cdff67: start","job_status":"start","queue_duration_s":2.127342,"scheduling_latency_s":2.127359} gitlab # [ 450.735142] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a86df3d92a8bb7355cdff67","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a86df3d92a8bb7355cdff67: done: 0.017225 sec","job_status":"done","queue_duration_s":2.127342,"scheduling_latency_s":2.127359,"redis_calls":13,"redis_duration_s":0.003401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00243,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017225,"completed_at":"2026-08-27T02:08:47.111Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.775910] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37fca2e13fd22290efc1d839","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37fca2e13fd22290efc1d839: start","job_status":"start","queue_duration_s":2.147646,"scheduling_latency_s":2.147666} gitlab # [ 450.788882] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37fca2e13fd22290efc1d839","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37fca2e13fd22290efc1d839: done: 0.017177 sec","job_status":"done","queue_duration_s":2.147646,"scheduling_latency_s":2.147666,"redis_calls":13,"redis_duration_s":0.0027170000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017177,"completed_at":"2026-08-27T02:08:47.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 450.830012] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55ebac054a6cf5b3efcd434f","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55ebac054a6cf5b3efcd434f: start","job_status":"start","queue_duration_s":2.166164,"scheduling_latency_s":2.166181} gitlab # [ 450.868956] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55ebac054a6cf5b3efcd434f","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55ebac054a6cf5b3efcd434f: done: 0.017396 sec","job_status":"done","queue_duration_s":2.166164,"scheduling_latency_s":2.166181,"redis_calls":13,"redis_duration_s":0.002778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001852,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017396,"completed_at":"2026-08-27T02:08:47.150Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 450.948344] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e990aae698bd449fe5a3df1a","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e990aae698bd449fe5a3df1a: start","job_status":"start","queue_duration_s":2.185112,"scheduling_latency_s":2.185129} gitlab # [ 450.984769] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e990aae698bd449fe5a3df1a","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e990aae698bd449fe5a3df1a: done: 0.016875 sec","job_status":"done","queue_duration_s":2.185112,"scheduling_latency_s":2.185129,"redis_calls":13,"redis_duration_s":0.002903,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016875,"completed_at":"2026-08-27T02:08:47.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.058702] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"433ef4e2a4f038ed3807d30e","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-433ef4e2a4f038ed3807d30e: start","job_status":"start","queue_duration_s":2.20497,"scheduling_latency_s":2.204989} gitlab # [ 451.075140] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"433ef4e2a4f038ed3807d30e","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-433ef4e2a4f038ed3807d30e: done: 0.017733 sec","job_status":"done","queue_duration_s":2.20497,"scheduling_latency_s":2.204989,"redis_calls":13,"redis_duration_s":0.002869,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017733,"completed_at":"2026-08-27T02:08:47.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.148610] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54054850875659e48c68082c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54054850875659e48c68082c: start","job_status":"start","queue_duration_s":2.224281,"scheduling_latency_s":2.224297} gitlab # [ 451.171941] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54054850875659e48c68082c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54054850875659e48c68082c: done: 0.016668 sec","job_status":"done","queue_duration_s":2.224281,"scheduling_latency_s":2.224297,"redis_calls":13,"redis_duration_s":0.002768,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001809,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016668,"completed_at":"2026-08-27T02:08:47.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.222068] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3d07068698017d551b8ffe7","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3d07068698017d551b8ffe7: start","job_status":"start","queue_duration_s":2.243967,"scheduling_latency_s":2.243986} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.28 seconds) gitlab # [ 451.235921] gitaly[906]: time="2026-08-27T02:08:48.036Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FM6SXE8AE3MXD7CCQ0B48 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:18.034" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:48.034" grpc.time_ms=1.128 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 451.249217] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 154 gitlab # [ 451.251861] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3d07068698017d551b8ffe7","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3d07068698017d551b8ffe7: done: 0.017461 sec","job_status":"done","queue_duration_s":2.243967,"scheduling_latency_s":2.243986,"redis_calls":13,"redis_duration_s":0.0038789999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017461,"completed_at":"2026-08-27T02:08:47.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.292772] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2e6aa922be37c0c22409124","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2e6aa922be37c0c22409124: start","job_status":"start","queue_duration_s":2.262966,"scheduling_latency_s":2.262983} gitlab # [ 451.305512] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2e6aa922be37c0c22409124","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2e6aa922be37c0c22409124: done: 0.016822 sec","job_status":"done","queue_duration_s":2.262966,"scheduling_latency_s":2.262983,"redis_calls":13,"redis_duration_s":0.002987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016822,"completed_at":"2026-08-27T02:08:47.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.346742] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"581b4ef35312a7a2db0f1701","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-581b4ef35312a7a2db0f1701: start","job_status":"start","queue_duration_s":2.282147,"scheduling_latency_s":2.282166} gitlab # [ 451.359630] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"581b4ef35312a7a2db0f1701","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-581b4ef35312a7a2db0f1701: done: 0.016889 sec","job_status":"done","queue_duration_s":2.282147,"scheduling_latency_s":2.282166,"redis_calls":13,"redis_duration_s":0.0027990000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016889,"completed_at":"2026-08-27T02:08:47.265Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.400434] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be4bd17e20faa7bca2261616","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be4bd17e20faa7bca2261616: start","job_status":"start","queue_duration_s":2.300577,"scheduling_latency_s":2.300594} gitlab # [ 451.413104] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be4bd17e20faa7bca2261616","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be4bd17e20faa7bca2261616: done: 0.016458 sec","job_status":"done","queue_duration_s":2.300577,"scheduling_latency_s":2.300594,"redis_calls":13,"redis_duration_s":0.002837,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016458,"completed_at":"2026-08-27T02:08:47.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.452942] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16b2aa7131399b21af5dcd53","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16b2aa7131399b21af5dcd53: start","job_status":"start","queue_duration_s":2.319127,"scheduling_latency_s":2.31915} gitlab # [ 451.465637] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16b2aa7131399b21af5dcd53","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16b2aa7131399b21af5dcd53: done: 0.016797 sec","job_status":"done","queue_duration_s":2.319127,"scheduling_latency_s":2.31915,"redis_calls":13,"redis_duration_s":0.003041,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016797,"completed_at":"2026-08-27T02:08:47.301Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.506177] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9a05b794d3a6195a3f3521e","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9a05b794d3a6195a3f3521e: start","job_status":"start","queue_duration_s":2.336923,"scheduling_latency_s":2.33694} gitlab # [ 451.519011] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9a05b794d3a6195a3f3521e","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9a05b794d3a6195a3f3521e: done: 0.018163 sec","job_status":"done","queue_duration_s":2.336923,"scheduling_latency_s":2.33694,"redis_calls":13,"redis_duration_s":0.003151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018163,"completed_at":"2026-08-27T02:08:47.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.562036] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30dda3eb46af0b9462091e85","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30dda3eb46af0b9462091e85: start","job_status":"start","queue_duration_s":2.356703,"scheduling_latency_s":2.356719} gitlab # [ 451.575040] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30dda3eb46af0b9462091e85","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30dda3eb46af0b9462091e85: done: 0.01706 sec","job_status":"done","queue_duration_s":2.356703,"scheduling_latency_s":2.356719,"redis_calls":13,"redis_duration_s":0.0028469999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01706,"completed_at":"2026-08-27T02:08:47.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.615898] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2854350ea0513eba0d3daa8","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2854350ea0513eba0d3daa8: start","job_status":"start","queue_duration_s":2.375299,"scheduling_latency_s":2.375316} gitlab # [ 451.628987] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2854350ea0513eba0d3daa8","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2854350ea0513eba0d3daa8: done: 0.017298 sec","job_status":"done","queue_duration_s":2.375299,"scheduling_latency_s":2.375316,"redis_calls":13,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017298,"completed_at":"2026-08-27T02:08:47.359Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.669690] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"493ea2e45030df3c9458bd40","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-493ea2e45030df3c9458bd40: start","job_status":"start","queue_duration_s":2.394131,"scheduling_latency_s":2.394148} gitlab # [ 451.682500] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"493ea2e45030df3c9458bd40","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-493ea2e45030df3c9458bd40: done: 0.016472 sec","job_status":"done","queue_duration_s":2.394131,"scheduling_latency_s":2.394148,"redis_calls":13,"redis_duration_s":0.003005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016472,"completed_at":"2026-08-27T02:08:47.377Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.723124] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30e1f7a0dec483fbb765b7f5","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30e1f7a0dec483fbb765b7f5: start","job_status":"start","queue_duration_s":2.412063,"scheduling_latency_s":2.41208} gitlab # [ 451.737048] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30e1f7a0dec483fbb765b7f5","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30e1f7a0dec483fbb765b7f5: done: 0.016722 sec","job_status":"done","queue_duration_s":2.412063,"scheduling_latency_s":2.41208,"redis_calls":13,"redis_duration_s":0.002729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001793,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016722,"completed_at":"2026-08-27T02:08:47.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.777481] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90a27485875c6bbff894eb4d","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90a27485875c6bbff894eb4d: start","job_status":"start","queue_duration_s":2.43079,"scheduling_latency_s":2.430807} gitlab # [ 451.790289] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90a27485875c6bbff894eb4d","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90a27485875c6bbff894eb4d: done: 0.016997 sec","job_status":"done","queue_duration_s":2.43079,"scheduling_latency_s":2.430807,"redis_calls":13,"redis_duration_s":0.0034790000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016997,"completed_at":"2026-08-27T02:08:47.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.830984] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77e25c52e1e698e4649cd880","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77e25c52e1e698e4649cd880: start","job_status":"start","queue_duration_s":2.449371,"scheduling_latency_s":2.449389} gitlab # [ 451.843954] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77e25c52e1e698e4649cd880","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77e25c52e1e698e4649cd880: done: 0.01696 sec","job_status":"done","queue_duration_s":2.449371,"scheduling_latency_s":2.449389,"redis_calls":13,"redis_duration_s":0.002716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001752,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01696,"completed_at":"2026-08-27T02:08:47.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.884786] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5b2f85ec8bf32d79cd986eb","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5b2f85ec8bf32d79cd986eb: start","job_status":"start","queue_duration_s":2.467951,"scheduling_latency_s":2.467967} gitlab # [ 451.897555] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5b2f85ec8bf32d79cd986eb","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5b2f85ec8bf32d79cd986eb: done: 0.016158 sec","job_status":"done","queue_duration_s":2.467951,"scheduling_latency_s":2.467967,"redis_calls":13,"redis_duration_s":0.002881,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016158,"completed_at":"2026-08-27T02:08:47.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.938835] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a04ce47204c145d47290610","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a04ce47204c145d47290610: start","job_status":"start","queue_duration_s":2.486345,"scheduling_latency_s":2.486363} gitlab # [ 451.951816] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a04ce47204c145d47290610","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a04ce47204c145d47290610: done: 0.016248 sec","job_status":"done","queue_duration_s":2.486345,"scheduling_latency_s":2.486363,"redis_calls":13,"redis_duration_s":0.002852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016248,"completed_at":"2026-08-27T02:08:47.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 451.992504] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caab46be81d9046638b7ec51","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caab46be81d9046638b7ec51: start","job_status":"start","queue_duration_s":2.504119,"scheduling_latency_s":2.504135} gitlab # [ 452.005137] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caab46be81d9046638b7ec51","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caab46be81d9046638b7ec51: done: 0.01607 sec","job_status":"done","queue_duration_s":2.504119,"scheduling_latency_s":2.504135,"redis_calls":13,"redis_duration_s":0.002913,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01607,"completed_at":"2026-08-27T02:08:47.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.045491] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d57fcf7ae20d6a82f35f134c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d57fcf7ae20d6a82f35f134c: start","job_status":"start","queue_duration_s":2.522144,"scheduling_latency_s":2.522163} gitlab # [ 452.058291] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d57fcf7ae20d6a82f35f134c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d57fcf7ae20d6a82f35f134c: done: 0.0162 sec","job_status":"done","queue_duration_s":2.522144,"scheduling_latency_s":2.522163,"redis_calls":13,"redis_duration_s":0.002819,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0162,"completed_at":"2026-08-27T02:08:47.504Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.098785] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0cd7d4c3dc95aacab7905687","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0cd7d4c3dc95aacab7905687: start","job_status":"start","queue_duration_s":2.539877,"scheduling_latency_s":2.539894} gitlab # [ 452.111677] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0cd7d4c3dc95aacab7905687","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0cd7d4c3dc95aacab7905687: done: 0.016679 sec","job_status":"done","queue_duration_s":2.539877,"scheduling_latency_s":2.539894,"redis_calls":13,"redis_duration_s":0.003088,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016679,"completed_at":"2026-08-27T02:08:47.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.153278] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"436be7d67507a59f3835dce2","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-436be7d67507a59f3835dce2: start","job_status":"start","queue_duration_s":2.559926,"scheduling_latency_s":2.559976} gitlab # [ 452.166078] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"436be7d67507a59f3835dce2","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-436be7d67507a59f3835dce2: done: 0.0404 sec","job_status":"done","queue_duration_s":2.559926,"scheduling_latency_s":2.559976,"redis_calls":13,"redis_duration_s":0.00492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001655,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0404,"completed_at":"2026-08-27T02:08:47.566Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.206583] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef73e2589d646fcb812c94d1","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef73e2589d646fcb812c94d1: start","job_status":"start","queue_duration_s":2.603361,"scheduling_latency_s":2.603399} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 452.219508] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef73e2589d646fcb812c94d1","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef73e2589d646fcb812c94d1: done: 0.031516 sec","job_status":"done","queue_duration_s":2.603361,"scheduling_latency_s":2.603399,"redis_calls":13,"redis_duration_s":0.003529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031516,"completed_at":"2026-08-27T02:08:47.601Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.259017] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c880914ad2dbc7da885ada","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c880914ad2dbc7da885ada: start","job_status":"start","queue_duration_s":2.636946,"scheduling_latency_s":2.636971} gitlab # [ 452.270893] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c880914ad2dbc7da885ada","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c880914ad2dbc7da885ada: done: 0.023946 sec","job_status":"done","queue_duration_s":2.636946,"scheduling_latency_s":2.636971,"redis_calls":13,"redis_duration_s":0.003913,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023946,"completed_at":"2026-08-27T02:08:47.627Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.310960] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3eb9430fa8e35b21950ff151","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3eb9430fa8e35b21950ff151: start","job_status":"start","queue_duration_s":2.663354,"scheduling_latency_s":2.663379} gitlab # [ 452.323941] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3eb9430fa8e35b21950ff151","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3eb9430fa8e35b21950ff151: done: 0.02265 sec","job_status":"done","queue_duration_s":2.663354,"scheduling_latency_s":2.663379,"redis_calls":13,"redis_duration_s":0.003209,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02265,"completed_at":"2026-08-27T02:08:47.652Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.365378] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"757fe34b669659e543b30677","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-757fe34b669659e543b30677: start","job_status":"start","queue_duration_s":2.687864,"scheduling_latency_s":2.687885} gitlab # [ 452.378213] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"757fe34b669659e543b30677","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-757fe34b669659e543b30677: done: 0.019732 sec","job_status":"done","queue_duration_s":2.687864,"scheduling_latency_s":2.687885,"redis_calls":13,"redis_duration_s":0.003096,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019732,"completed_at":"2026-08-27T02:08:47.674Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.418685] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2f509ea7201cbedcd5ddc11","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2f509ea7201cbedcd5ddc11: start","job_status":"start","queue_duration_s":2.709424,"scheduling_latency_s":2.709444} gitlab # [ 452.431637] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2f509ea7201cbedcd5ddc11","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2f509ea7201cbedcd5ddc11: done: 0.018008 sec","job_status":"done","queue_duration_s":2.709424,"scheduling_latency_s":2.709444,"redis_calls":13,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018008,"completed_at":"2026-08-27T02:08:47.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.472631] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e70a39f17fd02458c14a002f","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e70a39f17fd02458c14a002f: start","job_status":"start","queue_duration_s":2.729095,"scheduling_latency_s":2.729116} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.29 seconds) gitlab # [ 452.485361] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e70a39f17fd02458c14a002f","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e70a39f17fd02458c14a002f: done: 0.016418 sec","job_status":"done","queue_duration_s":2.729095,"scheduling_latency_s":2.729116,"redis_calls":13,"redis_duration_s":0.002617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016418,"completed_at":"2026-08-27T02:08:47.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.525602] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bd8901a269163a5bc2aff7b","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bd8901a269163a5bc2aff7b: start","job_status":"start","queue_duration_s":2.747375,"scheduling_latency_s":2.747395} gitlab # [ 452.538077] gitaly[906]: time="2026-08-27T02:08:49.331Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FM81YMXSB8F0XCAXWWBV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:19.329" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:49.329" grpc.time_ms=1.182 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 452.551647] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 175 gitlab # [ 452.555450] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bd8901a269163a5bc2aff7b","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bd8901a269163a5bc2aff7b: done: 0.016384 sec","job_status":"done","queue_duration_s":2.747375,"scheduling_latency_s":2.747395,"redis_calls":13,"redis_duration_s":0.002549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001626,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016384,"completed_at":"2026-08-27T02:08:47.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.596257] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de646e664153e97290d1d592","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de646e664153e97290d1d592: start","job_status":"start","queue_duration_s":2.765421,"scheduling_latency_s":2.765441} gitlab # [ 452.608909] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de646e664153e97290d1d592","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de646e664153e97290d1d592: done: 0.017003 sec","job_status":"done","queue_duration_s":2.765421,"scheduling_latency_s":2.765441,"redis_calls":13,"redis_duration_s":0.002724,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001722,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017003,"completed_at":"2026-08-27T02:08:47.748Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.649022] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b74c1da66e78d76e87c3137c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b74c1da66e78d76e87c3137c: start","job_status":"start","queue_duration_s":2.784021,"scheduling_latency_s":2.784039} gitlab # [ 452.661719] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b74c1da66e78d76e87c3137c","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b74c1da66e78d76e87c3137c: done: 0.016735 sec","job_status":"done","queue_duration_s":2.784021,"scheduling_latency_s":2.784039,"redis_calls":13,"redis_duration_s":0.002856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016735,"completed_at":"2026-08-27T02:08:47.767Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.702088] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7a429bcbed28ccbaa566eee","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7a429bcbed28ccbaa566eee: start","job_status":"start","queue_duration_s":2.802642,"scheduling_latency_s":2.802661} gitlab # [ 452.714891] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7a429bcbed28ccbaa566eee","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7a429bcbed28ccbaa566eee: done: 0.017546 sec","job_status":"done","queue_duration_s":2.802642,"scheduling_latency_s":2.802661,"redis_calls":13,"redis_duration_s":0.0035180000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00181,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001708,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017546,"completed_at":"2026-08-27T02:08:47.786Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.755562] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32c689d65af042ad0af52762","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32c689d65af042ad0af52762: start","job_status":"start","queue_duration_s":2.821893,"scheduling_latency_s":2.821913} gitlab # [ 452.769425] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32c689d65af042ad0af52762","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32c689d65af042ad0af52762: done: 0.017192 sec","job_status":"done","queue_duration_s":2.821893,"scheduling_latency_s":2.821913,"redis_calls":13,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017192,"completed_at":"2026-08-27T02:08:47.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.809707] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"add3104911fd5d6bab3bccd9","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-add3104911fd5d6bab3bccd9: start","job_status":"start","queue_duration_s":2.844019,"scheduling_latency_s":2.844043} gitlab # [ 452.822575] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"add3104911fd5d6bab3bccd9","created_at":"2026-08-27T02:08:44.355Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-65d6e39fd0654faf","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7f27ac8191a2e8816d44a175d41e4077","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:44.966Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-add3104911fd5d6bab3bccd9: done: 0.017786 sec","job_status":"done","queue_duration_s":2.844019,"scheduling_latency_s":2.844043,"redis_calls":13,"redis_duration_s":0.002432,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001566,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017786,"completed_at":"2026-08-27T02:08:47.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.863147] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:47.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e942004734e50da6a1efb624","created_at":"2026-08-27T02:08:44.987Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08b3ed200501ded15c761eaac0d1beb9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.018Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e942004734e50da6a1efb624: start","job_status":"start","queue_duration_s":2.811641,"scheduling_latency_s":2.811661} gitlab # [ 452.877800] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e942004734e50da6a1efb624","created_at":"2026-08-27T02:08:44.987Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08b3ed200501ded15c761eaac0d1beb9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.018Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e942004734e50da6a1efb624: done: 0.290587 sec","job_status":"done","queue_duration_s":2.811641,"scheduling_latency_s":2.811661,"redis_calls":13,"redis_duration_s":0.0035770000000000003,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001092,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.225135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00071314,"exclusive_lock_hold_duration_s":0.26242538,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.290587,"completed_at":"2026-08-27T02:08:48.120Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.073349,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.923856] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.122Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d9604f3f072fa8ec66764bd5","created_at":"2026-08-27T02:08:45.095Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d9e2343a51707f993e410bdb9b1593b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.123Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d9604f3f072fa8ec66764bd5: start","job_status":"start","queue_duration_s":2.998879,"scheduling_latency_s":2.9989} gitlab # [ 452.939091] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.137Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 452.943123] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.147Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d9604f3f072fa8ec66764bd5","created_at":"2026-08-27T02:08:45.095Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d9e2343a51707f993e410bdb9b1593b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.123Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d9604f3f072fa8ec66764bd5: done: 0.025254 sec","job_status":"done","queue_duration_s":2.998879,"scheduling_latency_s":2.9989,"redis_calls":9,"redis_duration_s":0.002447,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000604,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001321,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025254,"completed_at":"2026-08-27T02:08:48.147Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000864,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 452.987989] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"65287269930ae34d12877a20","created_at":"2026-08-27T02:08:45.144Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b426a03446962845c55543172341603c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.169Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-65287269930ae34d12877a20: start","job_status":"start","queue_duration_s":2.980424,"scheduling_latency_s":2.980449} gitlab # [ 453.002986] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.164Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 453.006572] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.174Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"65287269930ae34d12877a20","created_at":"2026-08-27T02:08:45.144Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b426a03446962845c55543172341603c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.169Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-65287269930ae34d12877a20: done: 0.024594 sec","job_status":"done","queue_duration_s":2.980424,"scheduling_latency_s":2.980449,"redis_calls":9,"redis_duration_s":0.002407,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000568,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000534,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024594,"completed_at":"2026-08-27T02:08:48.174Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000743,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 453.050410] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.177Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"faef87ac369691c2de1d2b72","created_at":"2026-08-27T02:08:45.199Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a70eddb54c73de895dfbf04ebadb21c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.223Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-faef87ac369691c2de1d2b72: start","job_status":"start","queue_duration_s":2.953498,"scheduling_latency_s":2.953525} gitlab # [ 453.065064] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.192Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 453.068205] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"faef87ac369691c2de1d2b72","created_at":"2026-08-27T02:08:45.199Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a70eddb54c73de895dfbf04ebadb21c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.223Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-faef87ac369691c2de1d2b72: done: 0.025067 sec","job_status":"done","queue_duration_s":2.953498,"scheduling_latency_s":2.953525,"redis_calls":9,"redis_duration_s":0.002352,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000543,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001281,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025067,"completed_at":"2026-08-27T02:08:48.202Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 453.111667] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6cb8f3a82ed1a3dc92275f91","created_at":"2026-08-27T02:08:45.256Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6227dcf8a1ac757710360ef3ac3921d5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.269Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6cb8f3a82ed1a3dc92275f91: start","job_status":"start","queue_duration_s":2.934957,"scheduling_latency_s":2.935013} gitlab # [ 453.126872] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.276Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6cb8f3a82ed1a3dc92275f91","created_at":"2026-08-27T02:08:45.256Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6227dcf8a1ac757710360ef3ac3921d5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.269Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6cb8f3a82ed1a3dc92275f91: done: 0.072034 sec","job_status":"done","queue_duration_s":2.934957,"scheduling_latency_s":2.935013,"redis_calls":9,"redis_duration_s":0.004675,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000929,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002656,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.072034,"completed_at":"2026-08-27T02:08:48.276Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007127,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 453.171371] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.280Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"0231eac8019eacc6b4580b4c","created_at":"2026-08-27T02:08:45.287Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3bca17ec4b56329ca9bbaacf8853be8d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.298Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-0231eac8019eacc6b4580b4c: start","job_status":"start","queue_duration_s":2.981333,"scheduling_latency_s":2.981373} gitlab # [ 453.186668] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.324Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"0231eac8019eacc6b4580b4c","created_at":"2026-08-27T02:08:45.287Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3bca17ec4b56329ca9bbaacf8853be8d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.298Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-0231eac8019eacc6b4580b4c: done: 0.044172 sec","job_status":"done","queue_duration_s":2.981333,"scheduling_latency_s":2.981373,"redis_calls":11,"redis_duration_s":0.003838,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001364,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000713,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044172,"completed_at":"2026-08-27T02:08:48.324Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 453.230653] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"284f2be9b1575101a11a7fe3","created_at":"2026-08-27T02:08:45.367Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5b9fb62b323350ab8d327a91529770d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.411Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-284f2be9b1575101a11a7fe3: start","job_status":"start","queue_duration_s":2.915334,"scheduling_latency_s":2.915366} gitlab # [ 453.245662] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"284f2be9b1575101a11a7fe3","created_at":"2026-08-27T02:08:45.367Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-473b803500a84d81","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5b9fb62b323350ab8d327a91529770d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.411Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-284f2be9b1575101a11a7fe3: done: 0.030127 sec","job_status":"done","queue_duration_s":2.915334,"scheduling_latency_s":2.915366,"redis_calls":9,"redis_duration_s":0.002904,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000721,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001583,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0006,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030127,"completed_at":"2026-08-27T02:08:48.356Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 453.289581] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.359Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e74073679eda8769931e5435","created_at":"2026-08-27T02:08:45.798Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-0e29d287525c4f38","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.815Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e74073679eda8769931e5435: start","job_status":"start","queue_duration_s":2.543506,"scheduling_latency_s":2.543534} gitlab # [ 453.302287] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.382Z","message":"No existing merge request to be cleaned up."} gitlab # [ 453.303994] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.400Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e74073679eda8769931e5435","created_at":"2026-08-27T02:08:45.798Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-0e29d287525c4f38","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.815Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e74073679eda8769931e5435: done: 0.041334 sec","job_status":"done","queue_duration_s":2.543506,"scheduling_latency_s":2.543534,"redis_calls":12,"redis_duration_s":0.003757,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000587,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001302,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001868,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041334,"completed_at":"2026-08-27T02:08:48.400Z","load_balancing_strategy":"primary","db_duration_s":0.006203,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 453.345332] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.402Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"014b762504f397ebe1c95eab","created_at":"2026-08-27T02:08:45.798Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-0e29d287525c4f38","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.815Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-014b762504f397ebe1c95eab: start","job_status":"start","queue_duration_s":2.586708,"scheduling_latency_s":2.586733} gitlab # [ 453.358017] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.415Z","message":"No existing merge request to be cleaned up."} gitlab # [ 453.359689] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.426Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"014b762504f397ebe1c95eab","created_at":"2026-08-27T02:08:45.798Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-0e29d287525c4f38","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.815Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-014b762504f397ebe1c95eab: done: 0.02429 sec","job_status":"done","queue_duration_s":2.586708,"scheduling_latency_s":2.586733,"redis_calls":10,"redis_duration_s":0.002719,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001516,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02429,"completed_at":"2026-08-27T02:08:48.426Z","load_balancing_strategy":"primary","db_duration_s":0.002507,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 453.400604] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.428Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b5845cf285312bcf626f1cbe","created_at":"2026-08-27T02:08:45.798Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-0e29d287525c4f38","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.815Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b5845cf285312bcf626f1cbe: start","job_status":"start","queue_duration_s":2.61276,"scheduling_latency_s":2.612785} gitlab # [ 453.413150] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.441Z","message":"No existing merge request to be cleaned up."} gitlab # [ 453.414842] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.451Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b5845cf285312bcf626f1cbe","created_at":"2026-08-27T02:08:45.798Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-0e29d287525c4f38","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.815Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b5845cf285312bcf626f1cbe: done: 0.023091 sec","job_status":"done","queue_duration_s":2.61276,"scheduling_latency_s":2.612785,"redis_calls":10,"redis_duration_s":0.00269,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001134,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001556,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023091,"completed_at":"2026-08-27T02:08:48.451Z","load_balancing_strategy":"primary","db_duration_s":0.00278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 453.454588] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.453Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9f1583a17a24f1f899ad21ad","created_at":"2026-08-27T02:08:45.798Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-0e29d287525c4f38","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.815Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9f1583a17a24f1f899ad21ad: start","job_status":"start","queue_duration_s":2.637513,"scheduling_latency_s":2.637535} gitlab # [ 453.467327] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.465Z","message":"No existing merge request to be cleaned up."} gitlab # [ 453.469104] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:48.474Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9f1583a17a24f1f899ad21ad","created_at":"2026-08-27T02:08:45.798Z","trace_propagation_headers":{"sentry-trace":"f8e3a63bd6a64ab2b66373f23b1ed45b-0e29d287525c4f38","baggage":"sentry-trace_id=f8e3a63bd6a64ab2b66373f23b1ed45b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8a9e5a08e98f78cee67b5ea6b1be0eaa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-27T02:08:45.815Z","job_size_bytes":2,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9f1583a17a24f1f899ad21ad: done: 0.021712 sec","job_status":"done","queue_duration_s":2.637513,"scheduling_latency_s":2.637535,"redis_calls":10,"redis_duration_s":0.002636,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001504,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021712,"completed_at":"2026-08-27T02:08:48.474Z","load_balancing_strategy":"primary","db_duration_s":0.002755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 453.735179] gitaly[906]: time="2026-08-27T02:08:50.605Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FM99Y22ZJMB5H890AQHC3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:20.603" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:50.603" grpc.time_ms=1.295 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.26 seconds) gitlab # [ 453.785281] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 161 gitlab # [ 454.473838] gitaly[906]: time="2026-08-27T02:08:51.345Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:21.342" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:51.343" grpc.time_ms=1.4 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 454.590418] gitaly[906]: time="2026-08-27T02:08:51.461Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:09:01.291" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:51.442" grpc.time_ms=19.744 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 454.699779] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:51.569Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"14e6fa113931942d77a1d41c","created_at":"2026-08-27T02:08:51.551Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-63aed5df8e534d85","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:51.565Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-14e6fa113931942d77a1d41c: start","job_status":"start","queue_duration_s":0.003811,"scheduling_latency_s":0.003881} gitlab # [ 454.765823] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 436 gitlab # [ 454.775229] nixbot[3632]: {"ts": "2026-08-27T02:08:51.638Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/029b33b836639f8d58dd1784db28a8cf810aca0d \"HTTP/1.1 201 Created\""} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 454.922518] gitaly[906]: time="2026-08-27T02:08:51.793Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMADWREPWA5WPZVP9GEQ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:21.791" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:51.791" grpc.time_ms=1.149 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 454.986406] gitaly[906]: time="2026-08-27T02:08:51.857Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:51.856" grpc.time_ms=1.33 pid=906 span.kind=server system=grpc gitlab # [ 455.020966] gitaly[906]: time="2026-08-27T02:08:51.892Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMAG6YPYNHYN640ZGR54T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:21.889" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:51.890" grpc.time_ms=1.35 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.057237] gitaly[906]: time="2026-08-27T02:08:51.928Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:51.927" grpc.time_ms=1.065 pid=906 span.kind=server system=grpc gitlab # [ 455.075985] gitaly[906]: time="2026-08-27T02:08:51.947Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FMADWREPWA5WPZVP9GEQ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:09:01.222" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:51.933" grpc.time_ms=13.833 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.101932] gitaly[906]: time="2026-08-27T02:08:51.973Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:51.973" grpc.time_ms=0.527 pid=906 span.kind=server system=grpc gitlab # [ 455.116415] gitaly[906]: time="2026-08-27T02:08:51.988Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:21.789" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:51.790" operations="[repack_refs]" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.131289] gitaly[906]: time="2026-08-27T02:08:51.992Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG diskcache=9751b660-5adb-4600-8112-e5481de2204d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:21.789" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:51.790" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.144795] gitaly[906]: time="2026-08-27T02:08:51.992Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=164 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7056 command.oublock=0 command.real_time_ms=193 command.spawn_token_fork_ms=1 command.system_time_ms=80 command.user_time_ms=84 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:21.789" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:51.790" grpc.time_ms=201.544 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.161702] gitaly[906]: time="2026-08-27T02:08:51.995Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:21.789" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:51.790" pid=906 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-08-27 02:06:58.3158253 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.39 seconds) gitlab # [ 455.175997] gitaly[906]: time="2026-08-27T02:08:52.028Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:21.789" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:51.790" pid=906 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 455.191597] gitaly[906]: time="2026-08-27T02:08:52.028Z" level=info msg="optimized repository" clean_stale_data_time=32.50774ms clean_work_trees_time="152.16µs" commit_graph_time=620ns correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.8µs" pid=906 prune_time=260ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="2.58µs" storage_name=default system=housekeeping total_time=32.71248ms gitlab # [ 455.198459] gitaly[906]: time="2026-08-27T02:08:52.028Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:21.789" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:51.790" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.209329] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 671 "" "curl/8.21.0" 329 gitlab # [ 455.226993] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 453 gitlab # [ 455.230403] nixbot[3632]: {"ts": "2026-08-27T02:08:52.100Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/029b33b836639f8d58dd1784db28a8cf810aca0d \"HTTP/1.1 201 Created\""} gitlab # [ 455.249960] gitaly[906]: time="2026-08-27T02:08:52.121Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-27T02:09:22.119" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:52.120" grpc.time_ms=1.163 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.303577] gitaly[906]: time="2026-08-27T02:08:52.174Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:22.172" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:52.173" grpc.time_ms=1.114 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.340495] gitaly[906]: time="2026-08-27T02:08:52.205Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMAWMFT1WAY7TK0VP449Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:22.203" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:52.204" grpc.time_ms=0.978 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.366011] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.190Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"601f86d27a4b5005add3e4a7","created_at":"2026-08-27T02:08:52.186Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"14e6fa113931942d77a1d41c","job_size_bytes":3,"pid":3569,"job_status":"deduplicated","message":"PipelineProcessWorker JID-601f86d27a4b5005add3e4a7: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 455.389108] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.192Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"14e6fa113931942d77a1d41c","created_at":"2026-08-27T02:08:51.551Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-63aed5df8e534d85","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:51.565Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3569,"job_status":"rescheduled","message":"PipelineProcessWorker JID-14e6fa113931942d77a1d41c: rescheduled"} gitlab # [ 455.408183] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.202Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"14e6fa113931942d77a1d41c","created_at":"2026-08-27T02:08:51.551Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-63aed5df8e534d85","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:51.565Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-14e6fa113931942d77a1d41c: done: 0.633716 sec","job_status":"done","queue_duration_s":0.003811,"scheduling_latency_s":0.003881,"gitaly_calls":3,"gitaly_duration_s":0.218056,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.029788000000000002,"redis_read_bytes":239,"redis_write_bytes":21781,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002254,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001865,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.009261,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14466,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.008876,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000582,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001748,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002948,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.067,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.027,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.320321,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.53842298,"duration_s":0.633716,"completed_at":"2026-08-27T02:08:52.202Z","load_balancing_strategy":"primary","db_duration_s":0.075899,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 455.467128] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4cf066b418d6539172502c4e","created_at":"2026-08-27T02:08:51.770Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:51.776Z","job_size_bytes":23,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4cf066b418d6539172502c4e: start","job_status":"start","queue_duration_s":0.428475,"scheduling_latency_s":0.428508} gitlab # [ 455.485823] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.266Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4cf066b418d6539172502c4e","created_at":"2026-08-27T02:08:51.770Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:51.776Z","job_size_bytes":23,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4cf066b418d6539172502c4e: done: 0.061148 sec","job_status":"done","queue_duration_s":0.428475,"scheduling_latency_s":0.428508,"redis_calls":7,"redis_duration_s":0.0015669999999999998,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061148,"completed_at":"2026-08-27T02:08:52.266Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013789,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 455.533333] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.268Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"58e93d5d61f76db25dc5b6db","created_at":"2026-08-27T02:08:52.004Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.010Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-58e93d5d61f76db25dc5b6db: start","job_status":"start","queue_duration_s":0.257846,"scheduling_latency_s":0.257877} gitlab # [ 455.550865] gitaly[906]: time="2026-08-27T02:08:52.288Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FMAWMFT1WAY7TK0VP449Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-27T02:09:02.270" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:52.270" grpc.time_ms=17.659 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 455.565650] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.319Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"58e93d5d61f76db25dc5b6db","created_at":"2026-08-27T02:08:52.004Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.010Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-58e93d5d61f76db25dc5b6db: done: 0.051193 sec","job_status":"done","queue_duration_s":0.257846,"scheduling_latency_s":0.257877,"redis_calls":7,"redis_duration_s":0.002419,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001733,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000686,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051193,"completed_at":"2026-08-27T02:08:52.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011003,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 455.609238] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.322Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c8066ee95713ab9b56640bbc","created_at":"2026-08-27T02:08:52.029Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.034Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c8066ee95713ab9b56640bbc: start","job_status":"start","queue_duration_s":0.28794,"scheduling_latency_s":0.287972} gitlab # [ 455.625529] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/029b33b836639f8d58dd1784db28a8cf810aca0d HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 253 gitlab # [ 455.628287] nixbot[3632]: {"ts": "2026-08-27T02:08:52.371Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/029b33b836639f8d58dd1784db28a8cf810aca0d \"HTTP/1.1 201 Created\""} gitlab # [ 455.632331] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.377Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c8066ee95713ab9b56640bbc","created_at":"2026-08-27T02:08:52.029Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.034Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c8066ee95713ab9b56640bbc: done: 0.055203 sec","job_status":"done","queue_duration_s":0.28794,"scheduling_latency_s":0.287972,"redis_calls":7,"redis_duration_s":0.0058909999999999995,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005285,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000606,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055203,"completed_at":"2026-08-27T02:08:52.377Z","load_balancing_strategy":"primary","db_duration_s":0.010425,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 455.674769] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.379Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"f864e5fb605612c5c715ad7b","created_at":"2026-08-27T02:08:52.054Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.058Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f864e5fb605612c5c715ad7b: start","job_status":"start","queue_duration_s":0.321548,"scheduling_latency_s":0.321581} gitlab # [ 455.692922] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.440Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"f864e5fb605612c5c715ad7b","created_at":"2026-08-27T02:08:52.054Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.058Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f864e5fb605612c5c715ad7b: done: 0.060159 sec","job_status":"done","queue_duration_s":0.321548,"scheduling_latency_s":0.321581,"redis_calls":10,"redis_duration_s":0.004155,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001828,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001644,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000683,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060159,"completed_at":"2026-08-27T02:08:52.439Z","load_balancing_strategy":"primary","db_duration_s":0.008219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 455.739055] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.442Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e53239d0d6fbaf130305fe94","created_at":"2026-08-27T02:08:52.059Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.062Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e53239d0d6fbaf130305fe94: start","job_status":"start","queue_duration_s":0.379371,"scheduling_latency_s":0.379403} gitlab # [ 455.757794] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.493Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e53239d0d6fbaf130305fe94","created_at":"2026-08-27T02:08:52.059Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.062Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e53239d0d6fbaf130305fe94: done: 0.050822 sec","job_status":"done","queue_duration_s":0.379371,"scheduling_latency_s":0.379403,"redis_calls":7,"redis_duration_s":0.0033280000000000002,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002579,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000749,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.050822,"completed_at":"2026-08-27T02:08:52.492Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 455.806080] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.495Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"372494f360e2a3e0c7811ac7","created_at":"2026-08-27T02:08:52.064Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.068Z","job_size_bytes":19,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-372494f360e2a3e0c7811ac7: start","job_status":"start","queue_duration_s":0.427115,"scheduling_latency_s":0.427144} gitlab # [ 455.824356] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.561Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"372494f360e2a3e0c7811ac7","created_at":"2026-08-27T02:08:52.064Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.068Z","job_size_bytes":19,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-372494f360e2a3e0c7811ac7: done: 0.066519 sec","job_status":"done","queue_duration_s":0.427115,"scheduling_latency_s":0.427144,"redis_calls":8,"redis_duration_s":0.003434,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002222,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066519,"completed_at":"2026-08-27T02:08:52.561Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008864,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 455.871791] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.564Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2825958e2a05851cd6b61006","created_at":"2026-08-27T02:08:52.069Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.073Z","job_size_bytes":66,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2825958e2a05851cd6b61006: start","job_status":"start","queue_duration_s":0.490458,"scheduling_latency_s":0.490492} gitlab # [ 455.889584] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.610Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2825958e2a05851cd6b61006","created_at":"2026-08-27T02:08:52.069Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.073Z","job_size_bytes":66,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2825958e2a05851cd6b61006: done: 0.047006 sec","job_status":"done","queue_duration_s":0.490458,"scheduling_latency_s":0.490492,"redis_calls":7,"redis_duration_s":0.002025,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000677,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047006,"completed_at":"2026-08-27T02:08:52.610Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 455.934025] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.613Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"63d002897093d5d6d0dbab8f","created_at":"2026-08-27T02:08:52.075Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.079Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-63d002897093d5d6d0dbab8f: start","job_status":"start","queue_duration_s":0.533491,"scheduling_latency_s":0.533524} gitlab # [ 455.951988] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 123 gitlab # [ 455.954270] gitlab-workhorse[824]: time="2026-08-27T02:08:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 455.956299] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 103 gitlab # [ 455.958727] gitaly[906]: time="2026-08-27T02:08:52.706Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M10FMBB8102CT31HAVRQS2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:52.686" grpc.time_ms=20.072 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 455.973527] gitaly[906]: time="2026-08-27T02:08:52.811Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M10FMBF67A6HPC093QQ48YJK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:52.799" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 455.984731] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.680Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"63d002897093d5d6d0dbab8f","created_at":"2026-08-27T02:08:52.075Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.079Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-63d002897093d5d6d0dbab8f: done: 0.06747 sec","job_status":"done","queue_duration_s":0.533491,"scheduling_latency_s":0.533524,"redis_calls":7,"redis_duration_s":0.002023,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06747,"completed_at":"2026-08-27T02:08:52.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 456.028775] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.683Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"9d1b11d9c3aa7d1678e6d687","created_at":"2026-08-27T02:08:52.088Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.091Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9d1b11d9c3aa7d1678e6d687: start","job_status":"start","queue_duration_s":0.591064,"scheduling_latency_s":0.591101} gitlab # [ 456.047308] gitaly[906]: time="2026-08-27T02:08:52.812Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMBF67A6HPC093QQ48YJK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:52.799" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 456.061761] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.749Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"9d1b11d9c3aa7d1678e6d687","created_at":"2026-08-27T02:08:52.088Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.091Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9d1b11d9c3aa7d1678e6d687: done: 0.066696 sec","job_status":"done","queue_duration_s":0.591064,"scheduling_latency_s":0.591101,"redis_calls":7,"redis_duration_s":0.002814,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002003,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000811,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066696,"completed_at":"2026-08-27T02:08:52.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 456.107275] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.754Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"429db6b6d0b72db84d399418","created_at":"2026-08-27T02:08:52.092Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.095Z","job_size_bytes":6,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-429db6b6d0b72db84d399418: start","job_status":"start","queue_duration_s":0.658723,"scheduling_latency_s":0.658774} gitlab # [ 456.123586] gitaly[906]: time="2026-08-27T02:08:52.813Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMBF67A6HPC093QQ48YJK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-27T02:08:52.799" grpc.time_ms=13.261 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 456.138790] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.798Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"429db6b6d0b72db84d399418","created_at":"2026-08-27T02:08:52.092Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.095Z","job_size_bytes":6,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-429db6b6d0b72db84d399418: done: 0.043601 sec","job_status":"done","queue_duration_s":0.658723,"scheduling_latency_s":0.658774,"redis_calls":7,"redis_duration_s":0.00482,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002106,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043601,"completed_at":"2026-08-27T02:08:52.797Z","load_balancing_strategy":"primary","db_duration_s":0.001346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses gitlab # [ 456.181508] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.800Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"27c9e67243368f17f3f87c4c","created_at":"2026-08-27T02:08:52.096Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.100Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-27c9e67243368f17f3f87c4c: start","job_status":"start","queue_duration_s":0.69999,"scheduling_latency_s":0.700028} gitlab # [ 456.197415] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.857Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"27c9e67243368f17f3f87c4c","created_at":"2026-08-27T02:08:52.096Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.100Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-27c9e67243368f17f3f87c4c: done: 0.057175 sec","job_status":"done","queue_duration_s":0.69999,"scheduling_latency_s":0.700028,"redis_calls":7,"redis_duration_s":0.002033,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000704,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057175,"completed_at":"2026-08-27T02:08:52.857Z","load_balancing_strategy":"primary","db_duration_s":0.008935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 456.242156] sidekiq-cluster[3569]: {"severity":"INFO","time":"2026-08-27T02:08:52.863Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"7ca2a90885763108cf0e6477","created_at":"2026-08-27T02:08:52.193Z","trace_propagation_headers":{"sentry-trace":"4fe3ce6d19414e75a5c2a80e8f83183c-65f4724f025a4e7f","baggage":"sentry-trace_id=4fe3ce6d19414e75a5c2a80e8f83183c,sentry-release=Unknown"},"correlation_id":"01M10FMA00GKP1RBYSQ0Z78BRG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-27T02:08:52.195Z","job_size_bytes":3,"pid":3569,"sidekiq_tid":"m8pl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7ca2a90885763108cf0e6477: start","job_status":"start","queue_duration_s":0.667868,"scheduling_latency_s":0.667904} gitlab # [ 456.257944] gitaly[906]: time="2026-08-27T02:08:53.081Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:53.080" grpc.time_ms=0.884 pid=906 span.kind=server system=grpc gitlab # [ 456.267900] gitaly[906]: time="2026-08-27T02:08:53.131Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:53.130" grpc.time_ms=1.02 pid=906 span.kind=server system=grpc gitlab # [ 456.305740] gitaly[906]: time="2026-08-27T02:08:53.177Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-27T02:08:53.176" grpc.time_ms=0.799 pid=906 span.kind=server system=grpc gitlab # [ 456.321569] gitaly[906]: time="2026-08-27T02:08:53.193Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG diskcache=84ce60c7-4928-442e-906a-f4dc2e662232 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:23.022" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:53.023" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 456.332415] gitaly[906]: time="2026-08-27T02:08:53.193Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=173 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7397 command.oublock=0 command.real_time_ms=163 command.spawn_token_fork_ms=1 command.system_time_ms=107 command.user_time_ms=66 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-27T02:09:23.022" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-27T02:08:53.023" grpc.time_ms=170.389 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 456.378792] gitaly[906]: time="2026-08-27T02:08:53.250Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M10FMBWG968WS7249QF883FX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-27T02:09:23.248" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:53.248" grpc.time_ms=1.118 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses, in 0.27 seconds) gitlab # [ 456.437090] gitaly[906]: time="2026-08-27T02:08:53.308Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M10FMA00GKP1RBYSQ0Z78BRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-27T02:09:23.306" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-27T02:08:53.307" grpc.time_ms=1.338 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root (finished: subtest: eval and build post commit statuses, in 11.89 seconds) (finished: run the VM test script, in 529.06 seconds) gitlab # [ 456.456228] gitlab-workhorse[824]: gitlab 2001:db8:1::1 - - [2026/08/27:02:08:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/029b33b836639f8d58dd1784db28a8cf810aca0d/statuses HTTP/1.1" 200 1328 "" "curl/8.21.0" 178 test script finished in 529.10s cleanup kill QemuMachine (pid 46) gitlab # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.37 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-08-27T02:07:05.224Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-08-27T02:08:40.498923Z', 'started_at': '2026-08-27T02:08:40.920061Z', 'finished_at': None, 'eval_queue': None}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': '2026-08-27T02:08:51.525Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-08-27T02:08:40.869Z', 'started_at': None, 'finished_at': '2026-08-27T02:08:51.525Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '029b33b836639f8d58dd1784db28a8cf810aca0d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-08-27T02:08:52.028Z', 'started_at': None, 'finished_at': '2026-08-27T02:08:52.325Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]